Revisit Task Parallel Library (TPL)

The bot framework uses C# async, await, Task, etc. So I took this opportunity and revisited TPL.

My previous knowledge of TPL was from .net 4.0. It has been a while. So I did a warm up exercise. The source code is at the end of this blog.

When I ran the program, the control flow looked like:

  1. The main thread began
  2. The main thread kicked off task threads, and suspend. (UI screen unresponsive if you have UI). The task threads began, and finished one by one.
  3. The task threads waited for each other until all were done before transferred the control back to the main thread.
  4. The main thread ended

The reuse of threads from thread pools:

  1. If you let the threads slept short periods of time (simulating light workload), you will see there are only 3-4 worker threads running. The tasks reused threads from the pool
  2. If you let the threads slept longer periods of times (simulating heavy workload), you will see each task use its own thread.

 

References

TPL

Task

TaskFactory.StartNew

Task.WaitAll

 

Exercise

        
public class TPLDemo {
  public static void DemoRun() {
    Func<object, int> action = (object obj) => {
      int result = 0; // 0: fail, 1: success
      int workerNumber = (int) obj;

      try {
        // If you let the current thread sleep workNumber % 3 * 2000 s, 
        // you will see there are only 3-4 worker threads running
        int msSleep = workerNumber % 3 * 2000; 
        // If you let the current thread sleep 10s, 
        // you will see there are 10 worker threads running
        // int msSleep = 10000; 
        Console.WriteLine("Worker #" + workerNumber + ", ThreadId: " 
          + Thread.CurrentThread.ManagedThreadId 
          + ", begin to process tons of data here ... " + msSleep + " ms.");
        Thread.Sleep(msSleep);
        Console.WriteLine("Worker #" + workerNumber + ", ThreadId: " 
          + Thread.CurrentThread.ManagedThreadId + ", end");

        result = 1;
      } catch (Exception exc) {
        System.Console.WriteLine(exc.Message + ", " + exc.StackTrace);
      }

      return result;
    };

    int size = 10;
    Task[] tasks = new Task [size];
    for (int i = 0; i < size; i++) {
      tasks[i] = Task.Factory.StartNew(action, (i + 1));
    }

    try {
      Task.WaitAll(tasks);
      int sum = 0;
      for (int i = 0; i < size; i++) {
        sum += tasks[i].Result;
      }
      Console.WriteLine(sum);
    } catch (AggregateException e) {
      for (int j = 0; j < e.InnerExceptions.Count; j++)
        System.Console.WriteLine(e.InnerExceptions[j]);
        System.Console.WriteLine(e.Message + ", " + e.StackTrace);
      }
    }
}        
Advertisements

Bot Framework

Tags

,

We can see a lot of bots right now. More and more companies implemented some sort of bots to replace labor-intense tasks. They called Virtual Assistants. I remembered I used Aetna as my health insurance many years ago. Aetna used a Virtual Assistant, and I forgot her name. She can answer some simple questions, or present to you many choices if she cannot find the answers. The experience was similar to google search. Most bots we know belong to chat bots. Some bots have simple conversation capability with pattern matching technique, and some bots can tap into sophisticated artificial intelligence services.

I chose Microsoft Bot Framework due to my existing knowledge of Microsoft technology stack. I downloaded the bot framework, and created my echo back bot by following the tutorials.

It was quite easy to create a simple bot. The bot emulator was easy to use.

From the simple example I followed so far, I can feel we have to design a complex state machine to implement a more sophisticated bot…

References:

Microsoft Bot Framework

Create a bot with the Bot Builder SDK for .NET

 

LINQ’s Except Method #2 – IEqualityComparer

In LINQ’s Except Method #1, we went over GetHashCode in depth. In this blog, we will go over a special scenario on how to implement IEqualityComparer to be used in LINQ’s Except method.

public static IEnumerable Except(
    this IEnumerable first,
    IEnumerable second,
    IEqualityComparer comparer
)

MSDN provides an implementation reference [1]. To implement IEqualityComparer, we have to provide the implementation for Equals and GetHashCode. The example is enough for normal scenarios.

But what if we have this additional business requirement, we want to know the exact differences of products being compared in the same category, and don’t care to know the differences of products in different categories. For example, we know that Product { Name = “orange”, Code = 4} is in Except. In addition,

  1. We also want to know the difference between Product { Name = “orange”, Code = 4} of fruits1 and Product { Name = “orange”, Code = 8} of fruits2. These two products are in the same category.
  2. However, we don’t care the differences among orange and other fruits, and we also don’t want to waste resource to compare them.
            
            Product[] fruits1 = 
            {
                new Product { Name = "apple", Code = 9},
                new Product { Name = "orange", Code = 4},
                new Product { Name = "plum", Code = 4},
                new Product { Name = "apple", Code = 9},
                new Product { Name = "lemon", Code = 12}
            };

            Product[] fruits2 = 
            {                
                new Product { Name = "banana", Code = 11},
                new Product { Name = "lemon", Code = 14},
                new Product { Name = "orange", Code = 8},
                new Product { Name = "lemon", Code = 14},                
                new Product { Name = "apple", Code = 9},
                new Product { Name = "pineapple", Code = 12}
            };

In this case, we have to modify GetHashCode and Equals methods.

        // If Equals() returns true for a pair of objects 
        // then GetHashCode() must return the same value for these objects.
        public int GetHashCode(Product product)
        {            
             return product.Name.GetHashCode();        
        }

By using the product.Name’s hash code, we try to balance the distribution of each bucket. If they are the same products, the comparison then will use Equals to check further conditions, and satisfy the additional business requirement.

For the full implementation of Equals and exploration of Except / IEqualityComparer, please refer to this link in my GitHub. I suggest you down load the code, run, change the codes, and find out what’s really going on. There is tons of information I skip due to time constraints.

References:

  1. Enumerable.Except

Search Engine Optimization (SEO)- Robots and Search Console

Tags

,

If you want to improve SEO and give those crawler robots directives on how to search your website, you can set it up through Google Search Console.

In Google Search Console, you can claim your website as a property, create / test / add robots.txt to it. You can also link your Search Console with Google Analytics.

In simplified terms, robots.txt is a text file that webmasters create to instruct search engine robots how to crawl pages on their websites. To learn more about Google Search Console and robots.txt, please click on links in references below.

References:

A Beginner’s Guide to Google Search Console

Learn about robots.txt files

Create a robots.txt file

Test your robots.txt with the robots.txt Tester

Robots.txt

Learn About Robots.txt with Interactive Examples

robot.txt The ultimate guide

 

LINQ’s Except Method #1 – GetHashCode

In order to understand LINQ’s Except method, we first need to understand GetHashCode method.

What is hash code?

  1. A hash code is a numeric value that is used to insert and identify an object in a hash-based collection such as the Dictionary<TKey, TValue> class, the Hashtable class, or a type derived from the DictionaryBase class. [1]

 

What is the most important properties of hash code?

  1. Two objects that are equal return hash codes that are equal.
  2. However, the reverse is not true: equal hash codes do not imply object equality, because different (unequal) objects can have identical hash codes.  [1]

 

Why do we need hash code?

  1. The GetHashCode method provides this hash code for algorithms that need quick checks of object equality.
  2. Providing a good hash function on a class can significantly affect the performance of adding those objects to a hash table.
    1. In a hash table with keys that provide a good implementation of a hash function, searching for an element takes constant time (for example, an O(1) operation).
    2. In a hash table with a poor implementation of a hash function, the performance of a search depends on the number of items in the hash table (for example, an O(n) operation, where n is the number of items in the hash table). [1]

 

What is the relationship between GetHashCode and LINQ?

  1. A lot of the LINQ sequence operators use hash tables internally. If you want your type to play well with LINQ, take care of its GetHashCode method. [2]

 

The relationship and call sequence between GetHashCode and Equals?

Your item will be used as a key in a dictionary, or HashSet<T>, etc – since this is used (in the absence of a custom IEqualityComparer<T>) to group items into buckets. If the hash-code for two items does not match, they may never be considered equal (Equals will simply never be called). [3]

The GetHashCode() method should reflect the Equals logic; the rules are [3]:

  • if two things are equal (Equals(...) == true) then they must return the same value for GetHashCode()
  • if the GetHashCode() is equal, it is not necessary for them to be the same; this is a collision, and Equals will be called to see if it is a real equality or not.

In other words, the hash code is used as a first-step in determining equality only, and should never be used as a de-facto determination as to whether two objects are equal.

  1. If two objects’ hash codes are not equal then they are treated as not equal.
  2. If they have the same hash code, then they should then be checked for actual equality. [5]

 

References:

  1. Object.GetHashCode Method ()
  2. Guidelines and rules for GetHashCode

  3. Why is it important to override GetHashCode when Equals method is overridden?

  4. Default implementation for Object.GetHashCode()

Network Service Account’s Two Credentials

NT AUTHORITY\Network Service machine account is a special system account with least privileges (limited user rights and permissions).

It can, of course, access local resources. When the Network Service account’s local credentials are used to authenticate to local resources. The credentials are of the form NT AUTHORITY\Network Service.

It also does have network credentials to access network resources. When the Network Service account’s network credentials are used to authenticate to network resources. The credentials are of the form {DomainName}\{AppServerName$}, where {DomainName} is the domain of {Server} and {AppServerName} is your app server name. Don’t forget there is a $ at the end of {DomainName}\{AppServerName$}.

When you try to search it in the network resources through active directory, you click “Object Types”, and make sure “Computers” is checked.

Sidebar

What is NT Authority? First I thought it is a special domain on the server to represent server itself because it’s treated as a domain sometimes. After searching and read a few articles, I found, this explanation, as quoted, “NT_AUTHORITY essentially refers to the Windows operating system itself. Or perhaps as ‘things the OS authorizes on your behalf'” [4]. This explanation perhaps is easier to be understood for us, non system Admins.

References:

  1. Network Service account accessing a folder share
  2. NetworkService account access to a shared folder (Wins 2012R2)
  3. How To: Use the Network Service Account to Access Resources in ASP.NET
  4. What is “NT AUTHORITY” and “NT SERVICE”
  5. what are NT AUTHORITY accounts?   

 

 

C# Operator implicit vs explicit

In some business scenarios, we don’t want the time portion of DateTime type. So implemented this type DateOnly to convert these two types, DateOnly and DateTime, both ways implicitly.

public class DateOnly
{
    public DateTime Date { get; private set; }

    public DateOnly(DateTime dateTime) 
    { 
	Date = dateTime.Date; 
    }
   
    // User-defined conversion from DateOnly to DateTime
    public static implicit operator DateTime(DateOnly dateOnly) {
        return dateOnly.Date;
    }

    // User-defined conversion from DateTime to DateOnly
    public static implicit operator DateOnly(DateTime dateTime) {
        return new DateOnly(dateTime);
    }
}

 

A few days later, I happened to browse Microsoft Docs, and found there was another operator explicit. We can the type DateOnly using explicit operator.

public class DateOnly
{
    public DateTime Date { get; private set; }

    public DateOnly(DateTime dateTime) 
    { 
        Date = dateTime.Date; 
    }
   
    // User-defined conversion from DateOnly to DateTime
    public static explicit operator DateTime(DateOnly dateOnly) {
        return dateOnly.Date;
    }   

    // User-defined conversion from DateTime to DateOnly
    public static explicit operator DateOnly(DateTime dateTime) {
        return new DateOnly(dateTime);
    }
}

 

Should we use implicit or explicit operator for DateOnly. According to docments, the differences are:

  implicit explicit
cast not required required

exceptions

never possible
lose information never possible

I’d choose implicit operator in this case.

Reference:

implicit

explicit

 

 

 

 

Agile Is Great Unless / Until …

Agile is very popular. It seems like it solves all our problems if we simply just say or use it. We can forget about the traditional problem solving process. Problem solving is the act of defining a problem; determining the cause of the problem; identifying, prioritizing and selecting alternatives for a solution; and implementing a solution.

Here is the Manifesto for Agile Software Development:

“We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.”

From the Manifesto above, I just cannot see how it solves all our problems magically. To me, agile is just one extra way to approach problem, and doesn’t conflict with traditional problem solving process. They solve the problems from different angles, and can go hand-in-hand with each other. They should both on your shelve of tools. Any methodology has its application domain, and agile is not an exception.

I found that if some teams focus too much on agile and without sufficient planning can have initial success on faster delivery, follow by slower delivery, and then by maintenance nightmare a few years later. At that time, the original “glory” development team was long gone. Technical debt must be paid by the current team, which usually is much smaller in number. So I complete one of possible sentences here. In this case, agile is great unless you are the one who maintain it.

References:

  1. Problem Solving
  2. Manifesto for Agile Software Development

Could Block-Level Replication Replace Database Backup?

Had an interesting talk with DBAs and System Admins. The topic was about whether Block-Level Replication replace database backup.

To simplify, traditional database backup provides point-in-time recovery naturally. Replication is just duplication of the same data, whether it is good or bad. In order for replication to provide similar features like database backup, we have to add other DR strategies.

Reference:

Backup and Replication: What is the Difference?

Backup vs Replication

Visual Studio Continued Executing Even After Stopped Debugging!

Debugged some complex business scenario in multiple applications, and found out the cause. In order to show the problem at that particular moment, I had to stop the Visual Studio debugger during the first repetition of a for loop. I examined the result. The result was not what I had expected. It was the result after all repetitions in the for loop were executed.

I did it the second time, and the result was still the same. It was after all repetitions in the for loop were executed. Strange!

Did research, and found nothing.

To speak the truth, I almost wanted to unplug the network cable at that moment.

Continued to search, and found this article, How to: Stop Debugging or Stop Execution, in MSDN. It solved my problem.

Normally, if the program is launched from Visual Studio, stop debugging terminates the process you are debugging.

However, if you use local IIS and attach to the process, instead of launching it from Visual Studio, the process continues running. This was the problem I met.

There are a few options to terminate all attaches processes:

  1. On the Debug menu, choose Terminate All.
  2. Kill all those processes in Task Manager
  3. Restart IIS

 

Reference:

How to: Stop Debugging or Stop Execution