C# LINQ Distinct

Tags

Debugged through a legacy C# LINQ query, and this query contained a Distinct operator. The query exhibited strange behaviors. It took me a few minutes to realize that Distinct was one of the deferred operations. [1]

Another much more interesting and important topic is that what happens when we use Distinct without supplying an IEqualityComparer comparer [1]. Let’s use an example to illustrate.

public static IEnumerable Distinct(
    this IEnumerable source
)
void Main()
{
   Product[] products = {            
           new Product { Name = "apple", Code = 9 }, 
           new Product { Name = "orange", Code = 4 }, 
           new Product { Name = "apple", Code = 9 }, 
           new Product { Name = "lemon", Code = 12 } };

   IEnumberable noduplicates = products.Distinct();
   
   foreach (var product in noduplicates)
      Console.WriteLine(product.Name + " " + product.Code 
         + ", hash code: " + product.GetHashCode());
}
public class Product 
{
   public string Name { get; set; }
   public int Code { get; set; }
}

The code produces the following output:

apple 9, hash code: 4143056
orange 4, hash code: 52219803
apple 9, hash code: 1080906
lemon 12, hash code: 48640813

Why does “apple 9” appear twice? Why are the hash codes for “apple 9” different?

The system returns a default equality comparer [3]. Without any special equality comparer method implemented on our class Product, the system basically depends first on Object.GetHashCode, and then on Object.Equals to find out whether two objects are equal or not.

For reference  types, if GetHashCode is not overridden, hash codes are computed by calling the Object.GetHashCode method of the base class, which computes a hash code based on an object’s reference. In our case, two product instances of “apple 9” have different object references, therefore, different hash codes, and “apple 9” appear twice.

If value types do not override GetHashCode, the ValueType.GetHashCode method of the base class uses reflection to compute the hash code based on the values of the type’s fields. In other words, value types whose fields have equal values have equal hash codes.

References:

  1. Enumerable.Distinct
  2. Object.GetHashCode
  3. EqualityComparer.Default Property
  4. Object.Equals
  5. GetHashCode and LINQ
Advertisements

LocalService vs NetworkService vs LocalSystem

Wrote down the notes after solved an access issue [1, 2, ,3, 4].

LocalService NetworkService LocalSystem
Predefinied local account Yes, local authenticated users Yes, local authenticated users Yes
Privileges in local computer minimal, run standard least-privileged services minimal, run standard privileged services Extensive, have access to most system objects. Most services do not need such a high privilege level. If your service does not need these privileges, and it is not an interactive service, consider using the LocalService account or the NetworkService account.
Password No No No
Token NT Authority\System, Builtin\Administrators
Network identity Anonymous user* Act as the computer on the networks, computer’s credentials (computer$). By default, the remote token contains SIDs for the Everyone and Authenticated Users groups. Act as the computer on the networks, computer’s credentials (computer$)
Note By default, the Local System Account is a member of the local Administrators group and is therefore a member of the sysadmin group in SQL Server.

ATTN: * – Anonymous user is not equivalent to everyone. Everyone means a group of all authenticated users.

 

References:

  1. https://msdn.microsoft.com/en-us/library/windows/desktop/ms684188(v=vs.85).aspx
  2. https://msdn.microsoft.com/en-us/library/windows/desktop/ms684272(v=vs.85).aspx
  3. https://msdn.microsoft.com/en-us/library/windows/desktop/ms684188(v=vs.85).aspx
  4. https://stackoverflow.com/questions/510170/the-difference-between-the-local-system-account-and-the-network-service-acco

Error CS0006 – Metadata file XYZ.dll Not Found

Met this metadata file XYZ.dll not found error a few weeks ago, and spent quite some on figuring out what was going on. Followed the thread [1, 2], tried most popular approaches, and still cannot fix it.

In the end, I began to try cold leads. Some users mentioned to check the output window [1, 2], as quoted

for any reason, did not show up in the error window. Because of that, the VS build system seemed to miss the error and tried to build depending projects, which in turn failed with the annoying metadata message … First look at your Output Window! …

I looked at the output window before, and didn’t find anything special. Anyway, it wouldn’t hurt to try again. With special attention to output window, I was surprised to find there was really an error message hidden among other regular messages in output window, followed this lead, and fixed the metadata file not found error.

References:

  1. https://stackoverflow.com/questions/1421862/metadata-file-dll-could-not-be-found
  2. https://stackoverflow.com/questions/44251030/vs-2017-metadata-file-dll-could-not-be-found/44258162

Testing Delay Signed Assemblies

When testing delay signed assemblies using VS test projects, we sometimes got security strong name validation errors. Why? According to [2], this should not happen, as quoted:

“So to give the developer a fighting chance to debug and test his code, he needs a way to make it look like the assembly has a strong name. Delay signing uses a temporary private key and puts data in his registry that records that this temporary key is a substitute for the real one. The CLR looks for this registry key when it checks the strong name, giving the okay when it finds it.

So everything works like it normally does. As long as it is done on the specific machine that delay-signed the assembly”

We can fix this problem [3] by x32 or x64 version of sn using the following line.

sn.exe -Vr *,PUBLIC_KEY_TOKEN_HERE

We can verify the entry in the registry at

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\StrongName\Verification

If above method also fails, we might have to use a technique called conditional configuration [1]

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
   <SignAssembly>true</SignAssembly>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
   <AssemblyOriginatorKeyFile>Xyz.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
   <DelaySign>true</DelaySign>
</PropertyGroup>

 

References:

1. https://blogs.msdn.microsoft.com/mcsuksoldev/2013/05/23/unit-testing-delay-signed-assemblies-on-the-hosted-tfs-build-service/

2. https://stackoverflow.com/questions/21902883/my-delay-signed-assembly-is-debugging-why-is-it-like-this

3. https://stackoverflow.com/questions/5784910/how-to-unit-testing-delay-sign-assemblies-using-nunit

4. https://bartwullems.blogspot.com/2016/08/

5. https://windows-hexerror.linestarve.com/0x8013141A

Think Beyond Visitor Pattern

According to GOF [1], Visitor pattern is:

“Represent an operation to be performed on elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.”

If we spend some time on Visitor pattern, we sooner or later will realize that the real purpose behind Visitor pattern is to separate operations from objects. If we don’t just satisfy with our current understanding and look further to the horizon for a deeper understanding, what do we find?

Here is my 2c.

What if we apply this Visitor pattern to database, let database tables to be the “objects”, we then can write queries and commands as the “operations” against the “objects”. Is this a wilder application of Visitor pattern?

Can you find an even wilder application of Visitor pattern? Let me know ….

References:

  1. Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley.  ISBN  0-201-63361-2
  2. https://en.wikipedia.org/wiki/Visitor_pattern
  3. https://sourcemaking.com/design_patterns/visitor
  4. http://www.oodesign.com/visitor-pattern.html
  5. http://www.dofactory.com/net/visitor-design-pattern

What is Cache Busting? What about Ctrl + F5?

Have you ever met browser caching issues that you have to ask your users to use Ctrl + F5 to fix the problem? Do you want to get rid of this user-unfriendly behavior?

There are many ways to prevent this issue from happening. I just mention the promising three here [3].

  • File name versioning (e.g. style.v2.css).
  • File path versioning (e.g. /v2/style.css)
  • Query strings (e.g. style.css?ver=2). It is probably adequate because Stackoverflow currently uses this technique, and there are millions of Stackoverflow users [1]. But some authors point out it doesn’t always work [4]

File name versioning and file path versions, by theory, will always work.

Our life can become much easier if we just let the framework to do the job.

If you use asp.net MVC 5, you are lucky because it’s built in with bundling feature. In addition, the bundle path points to a virtual path, and makes it easy to change this virtual path.

public static void RegisterBundles(BundleCollection bundles)
{
    string version = "{version}"; // version from app settings, etc
    bundles.Add(new ScriptBundle("~/bundles/ABC-" + version)
           .Include("~/Scripts/ABC1.js",
                    "~/Scripts/ABC2.js",
                    "~/Scripts/ABC3.js"));

    // Code removed for clarity.
    BundleTable.EnableOptimizations = true;
}

For how to use MVC 5 bundling feature in details, please refer to Bundling and Minification [9].

Background: I talked to Mike, a front-end expert. He mentioned there was a term for fighting these unwanted browser caching behavior, and the term is call “Cache Busting”. I used to search using the keywords such as “clean browser caches”. Now I can search by “cache busting” instead.

By the way, the approach of cleaning browser caches programmatically using Javascript is a dead-end road because of security concerns that browsers will never allow you to do so. [6]

References:

  1. https://stackoverflow.com/questions/9692665/cache-busting-via-params
  2. http://www.adopsinsider.com/ad-ops-basics/what-is-a-cache-buster-and-how-does-it-work/
  3. https://www.keycdn.com/support/what-is-cache-busting/
  4. https://css-tricks.com/strategies-for-cache-busting-css/
  5. https://stackoverflow.com/questions/8155064/how-to-programmatically-empty-browser-cache
  6. https://www.quora.com/Is-there-a-method-to-program-an-automatic-clearing-of-a-browser-cache-in-JavaScript
  7. https://www.aspsnippets.com/Articles/Using-Bundles-ScriptBundle-in-ASPNet-MVC-Razor.aspx
  8. https://stackoverflow.com/questions/11979718/how-can-i-specify-an-explicit-scriptbundle-include-order
  9. https://docs.microsoft.com/en-us/aspnet/mvc/overview/performance/bundling-and-minification
  10. http://www.tutorialsteacher.com/mvc/scriptbundle-mvc
  11. http://www.c-sharpcorner.com/article/bundling-in-asp-net-mvc-application/

Expressive Power of JavaScript In Implementing Visitor Pattern

Implementing the Visitor Pattern using JavaScript for fun during this weekend.

According to GOF [1], visitor pattern is:

“Represent an operation to be performed on elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.”

There are three key components when using visitor pattern:

  1. Element hierarchy, and
  2. Visitors, and
  3. Clients that use element hierarchy and visitors.

In my implementation, I have:

  1. Element hierarchy. abstract employee, full time and part time concrete employees
  2. Visitors. totalAmountSalaryVisitor, totalAmountPartTimeVisitor, averageCompensationVisitor
  3. Clients. Creates employees, add visitors, and get the results.

JavaScript is very expressive. If you want to add a ‘accept’ function to the employees array, you just add it. You don’t need to use a wrapper or extension method like class-oriented languages.

var employees = [];
employees.accept = function(visitor) { 
  this.forEach(function(element) {
    element.accept(visitor);
  });
};

Adding ‘accept’ method to the abstract employee just as simple.

var employee = function(spec) {
  var that = {};
  that.firstName = spec.firstName || "*";
  that.lastName = spec.lastName || "*";
  that.age = spec.age || 0;
  that.employmentType = spec.employmentType || "Employee"; 
  that.accept = function(visitor) {
    visitor.visit(that);
  };
  return that;
}

 

JavaScript is also class-free. You can just create visitor anytime and in any shape you want as long as it has a ‘visit’ method.

var totalAmountSalaryVisitor = {
  total: 0,
  visit: function(fullTimeEmployee) {
    this.total += fullTimeEmployee.salary || 0;
  }
};

Compared to the similar implementation in C# [5], the JavaScript version is more nature and more expressive, requires much less lines of codes, and, most importantly, more fun!

Full source code: https://thimbleprojects.org/yellowship/418220/ (You have to click the ‘Remix’ button on the right top corner to see the source code.)

References:

  1. Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley.  ISBN  0-201-63361-2
  2. https://en.wikipedia.org/wiki/Visitor_pattern
  3. https://sourcemaking.com/design_patterns/visitor
  4. http://www.oodesign.com/visitor-pattern.html
  5. http://www.dofactory.com/net/visitor-design-pattern

 

 

 

A Review of Murach’s PHP and MySQL (3rd edition)

 

The combination of PHP, MySQL, and Linus has been very popular among open-source web developers for almost 20 years now. I used this powerful combination around 2000, and later moved on to .net, Oracle, and SQL server due to the changes of jobs. Fast forward 13 years, around 2013, one of my friends decided to jump on the wagon of PHP and MySQL from .net shop. He said there were more remote opportunities in PHP and MySQL world, and there were latest features such as MVC pattern and objects in PHP as well.

The book begins with introductory text on PHP and MySQL, and builds a simple website using MVC pattern. Then the content is divided into two threads. The first thread dives deep into mastering PHP. It covers functions, objects, etc. The second thread focuses on MySQL programming including basic database design concepts. These two threads then merges into the most exciting final chapters in my opinion, real-world e-commerce website embedded a content / image management sub-system.

I think this book covers more than enough topics on both PHP and MySQL to kick start your learning experience if you just begin. You should follow along the examples in the book and do the exercises. The learning experience is quite rewarding because we are going to learn the following systems and tooling, PHP / MySQL (of course), NetBean IDE, Xdebug, XAMPP Control Panel, MySQL work bench, etc. For more experienced developers, this book can be a desktop reference or a fun read. I highly recommend this book.

The book review was first posted at Amazon.

 

 

C# Extension Method vs Instance Method, and JavaScript Equivalent

C# extension method gives us the illusion of adding an instance method on a 3rd party class.

While worked on adding extension methods to WEB API controller, I felt these extension methods cannot do too much without accessing controller’s instance methods. So I switched to inheritance. BaseApiController<T> was inherited from ApiController, and I also added template methods and function objects to achieve what I wanted to do.

So when do we use extension method and when do we use instance method?

Extension Method Instance Method
Inheritance No Yes
Add to base class without notice Yes No
Can access objects’ internals No Yes
Create a new type No Yes
Scope namespace with inheritance tree
Design pattern similarity Decorator (add new behavior with modifying the base class)
Extend sealed class Yes No

Another interesting observation is that in Javascript, we can add a method to an object directly to do the same thing without extension method and instance method. So in my personal opinion, JavaScript is more expressive than C# if we know what we are doing.

 

References:

  1. https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/extension-methods
  2. https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/how-to-implement-and-call-a-custom-extension-method

 

Read Request Content in Web API

In some scenarios, we must read the http request content as string, and parse the string ourselves.

public IHttpActionResult POST() 
{        
     string requestContent = ReadRequestContent(Request).Result;        
     return OK(requestContent);
}
private async Task<string> ReadRequestContentAsync(
                               HttpRequestMessage Request)
{        
     string result = await Request.Content.ReadAsStringAsync();        
     return result; 
}

 

References:

  1. https://docs.microsoft.com/en-us/aspnet/web-api/overview/getting-started-with-aspnet-web-api/action-results
  2. https://weblog.west-wind.com/posts/2013/dec/13/accepting-raw-request-body-content-with-aspnet-web-api?Page=2