Rant about Rant: Stop the Madness

Have you caught yourself bashing some “anonymous” developer’s code? Of course, you have – don’t deny it, and I bet you enjoyed it too! I, for one, am tired of this madness. I see this happening way too much. In company after company, in team after team.

Mirror Mirror

When you are doing it for fun, perhaps you can justify it to your humorous conscious. However,  when you are truly criticizing somebody else’s work I ask you to look at the mirror first. Ask yourself this simple question: what if that was you? Yeah, yeah that sounds corny.  But I am honestly tired of people complaining too much about others code, without thinking that they are capable of making same mistakes, producing same piece of “sh@t code”.

So let me take you back in time: remember the time when you were young and confident. You thought you can conquer the world with your wit and charm, and then you brought down the entire production server with a simple misuses of static variable (no that wasn’t me). I am sure you do remember your big mistakes. Also, I bet you remember the time when you cursed at some developers code and then realized it was yours -yep, I see you smirking there.

Hope by now you realized that when you are criticizing other people’s code, you are very well criticizing yourself. Some of you might say “so really who cares? What’s the harm?”.  So let’s talk about harm here, shall we?

1. Personal Conflicts

Remember that the “walls have ears”. Invariably, the person whose code you are bashing will come to know it, and trust me if they don’t show visible resentment and passionate hatred toward you, they are thinking of water-boarding you in their heads. Time will come when they will do the same to you right when you are ready for that promotion.

2.Team Moral

When you complain, it creates very negative environment. If you pay attention, you will realize people start taking long lunches, they start losing their enthusiasm…you know the rest. It is hard for us to realize that, but you impact other people’s behavior, especially when you are senior.

3. Waste of time

I don’t know what it is in human nature, but we like to complain. So when you start complaining everybody become passionately involved in the conversation. You can see others slowly getting up from their desks and joining your conversation, leaving all work behind.  Like a pack of nasty hyenas  circling around each other and shouting slurs and wishing death upon the creator of a piece of code.

Does this all sound like immense dramatization to you, or is this true. You tell me? Pay attention to faces of your colleagues as they complain, and tell me they don’t start looking like they “hyenas”.

 

 

 

Two Things to Make Testing Fun

t4-template-visual-studio

How many of you actually enjoy writing tests, probably not a lot, right? Even though you know testing is important.  For some of us, it just doesn’t feel right to spend time on creating tests than working on our world class application. I will not argue the merits of testing here, but I will tell you two things that will make testing fun:

1. Make your testing about learning new things

I think testing is the best playground for learning new features of a framework, language,  and more importantly of your own application. I will go to the extent and declare testing as the only good way to learn hidden features of your own application.

Also, it is the only area where you can be free of worry (in most cases, now don’t go and hit your production site with a heavy load test). You can break what you want. For example, if you haven’t played with reflection much, and worried reflection might slow your app down use it in your tests. Find a class with a lot of properties and use reflection to get those properties and test them all, something like this:

    var	fields= YourObject.GetType().GetFields();
    foreach (var field in fields){
             // Some action that populate all fields of your object
             //Assert whether they were populated correctly or not
     }

 

2. Automate test generations, be clever and imaginative

Here is something fun to poke around for test automation: T4 templates. If you haven’t heard of T4 templates then get ready for a fun ride. T4 templates are simple yet very powerful tool for code generation. To get started add a new file in your shinny Visual Studio solution  and your add windows filter by template keyword:

 

Choose a file name and press enter; now you will get following error. This is normal, so press OK and continue:

 

 

Use the T4 template file as you would use any other c# file because you will be writing c# code. There are only few template syntax you will have to really know, but the important thing is that you have full access to c# libraries and you can import your own.  Here few important things to know:

1. Including external assemblies. Add following to your header:

<#@ assembly Name="YourDll.dll" #>

2.  using name spaces:

<#@ import namespace="System" #>

3. Output directive (if you simply want to output some string for example):

<#=YourString #>

 

Now lets build a simple test with this. We can use our previous example of reflection and produce a test for each property:

  [TestFixture]
	public class TestAllFields
	{

		[Test]
		[Category("Fields")]
		public void _Should_Be_Populated_After_Calling_Method_()
		{
		//Call Your Method
		//Assert
		}

	}

Happy testing, hope that added some salt to your otherwise boring testing.

 

 

 

SolrNet to the Rescue – Why Use SolrNet?

Recently I had to write an entire Data Access layer, for a very large SOLR based application.  I started the project by using existing API built on top of SOLR Sharp. I quickly realized that something more intuitive and easy to use is needed. That’s when I turned my attention to SOLR Net.
I quickly realized that it is the answer to my problems. It is really easy to use, intuitive, and easily extendable. In fact, it rescued my project.  My project finished with clean maintainable code, and more importantly in a very reasonable amount of time.
I want to share with you two specifics attributes of SOLR net that  I think should help you start using it instead of other clients.

SOLR Net DSL

If you have ever worked with query building, either in SQL, SOLR…, you know the importance of having a good mental picture of your queries. I know this sounds trivial but when you are coding, it makes a huge difference.  Let’s take an example:

Problem:

You have to create a query for a product name “handbag” which has price 1000. In Psuedo Code you need something like this:

Price Field Is 100 AND ProductName Field is “Handbag”

Solution 1:
You could have an API that requires you to initialize some arbitrary class for a field and then assign values. Something like this:

var priceCriteria  = new PriceCriteriaForSolr();
priceCriteria.Type= Numeric;
priceCriteria.Value = 1000;
priceCriteria.IsRange=False;

var productNameCriteria  = new ProductCriteriaForSolr();
priceCriteria.Type= String;
priceCriteria.Value = “HandBag”;
priceCriteria.IsRange=False;

As you see above there are just too many steps to accomplish a simple task of querying two fields.  Imagine a dozen of this. It will become a nightmare to maintain. The code is just not fluent. Now let’s look at how you can do the same in Solr Net:

Solution 2:

Query.Field("Price").Is(1000) && Query.Field("ProductName").Is(“HandBag”)

Wow! That was easy, to both write and read. It just makes sense. As you probably realized above is the SOLR NET DSL in action.  Also, note that SOLR Net overloads the && operator (it also overloads || operator).  Here are a few more examples of the DSL:

Query.Field("Price").Between(10).And(10000)
Query.Field("Price").From<string>("1000").To("*")

Query.Field("City").In("Los Angeles", "Atlanta", "New York")

SOLR NET Extensibility

It is usually true:  as much as you like it, no one library can meet all your coding needs , nor should it try to. However, it could be flexible enough so it can be adopted for your needs. SOLR net follows this path. It is flexible. It has nice clean interfaces that  can be inherited and it uses dependency injection to initialize its internal core components, so you could have access to those components too.

For example, if you need to have access to low level parser, you can simply get it  like this:

var parser = ServiceLocator.Current.GetInstance<ISolrQueryResultParser<Document>>();

Read more on this here .

Let’s take a meatier example:

Problem:
You have customized SOLR response to return additional facet fields. How do we parse that data and put it inside SOLR net’s response facet object?

Solution:

1. Create a Parser that implements  ISolrResponseParser.

In our example, we  are looking for a node in SOLR reponse called “psedofacets”. By implementing ISolrResponseParser, we can get something like this (note that in 4.0 XDoc is used instead of XmlDocument):

public class PsuedoFacetParser : ISolrResponseParser
    {
//FOR SOLR NET 3.0
        public void Parse(XmlDocument xml, SolrQueryResults results)
        {
            var mainPsuedoFacetNode = xml.SelectSingleNode("response/lst[@name='pseudofacet']");
            if (mainPsuedoFacetNode != null)
            {
                if (results.FacetFields != null)
                {
                    foreach (XmlNode fieldNode in mainPsuedoFacetNode.ChildNodes)
                    {
                        var field = fieldNode.Attributes["name"].Value;
                        var c = new List>();
                        foreach (XmlNode facetNode in fieldNode.ChildNodes)
                        {
                            var nameAttr = facetNode.Attributes == null ? null : facetNode.Attributes["name"];
                            var key = nameAttr == null ? "" : nameAttr.Value;
                            var value = Convert.ToInt32(facetNode.InnerText);
                            c.Add(new KeyValuePair(key, value));
                        }
                        results.FacetFields.Add(field, c);
                    }
                }
            }
        }
//FOR SOLR NET 4.0
        public void Parse(System.Xml.Linq.XDocument xml, SolrQueryResults results)
        {
            var mainPsuedoFacetNode = xml.XPathSelectElement("response/lst[@name='pseudofacet']");
            if (mainPsuedoFacetNode != null)
            {
                if (results.FacetFields != null)
                {
                    foreach (XElement fieldNode in mainPsuedoFacetNode.Elements())
                    {
                        var field =(string) fieldNode.Attribute("name");
                        var c = new List>();

                        foreach (XElement facetNode in fieldNode.Elements())
                        {
                            var nameAttr = facetNode.Attributes().Count() ==0? null :( string) facetNode.Attribute("name");
                            var key = nameAttr == null ? "" : nameAttr;
                            var value = Convert.ToInt32(facetNode.Value);
                            c.Add(new KeyValuePair(key, value));
                        }
                        results.FacetFields.Add(field, c);
                    }
                }
            }
        }
    }

Above code simply matches the psudeofacet node and then iterate through key value pairs and then add the node values to FacetFields Object of SolrNet.

2. Pass it before initializing the connection.

Startup.Container.Register<ISolrResponseParser<YourSolrDoc>>(typeof(PsuedoFacetParser<YourSolrDoc>).FullName, c => new PsuedoFacetParser<YourSolrDoc>());
Startup.Init<YourSolrDoc>(loggingConnection);

We just saw fluent querying and and easily extendable SolrNet library. I hope that it helps you with your porject. Happy coding.

Additional Resources:

Here are couple of must have links if you want to get started with solrnet :

1.   Solrent Net Main Site
2.   Google Group (Excellent Resource, very helpful)

A Simple way to run Tests with Actual Parameters

In this post I am going to demonstrate how easy it is to save actual parameters from your application, and use them in a test.

Before you tell me why don’t you try Pex, let me just say I have, but I couldn’t make it work easily, and I needed actual parameters that my application uses.  Yes I gave up easily, but that’s because I thought this was way easier to engineer. Also, yes I can use mocks, but tell me  using actual parameters  doesn’t make you feel more confident about your tests?

Without further ado then: the idea is to just record actual parameters from you application in JSON form,  and then use that in a simple Integration/ Unit Test. There are simply three steps to this:

1. Create a generic method to write to an xml file some string values.
2. Access the xml file in your test.
3. De-serialize and Use.

1. Create a generic method to write to an xml file some string values.

Create a generic method that create simple xml with “Params” as root and “Param” as children. This is very straight forward implemenation using Linq to XML. The method creates a new if itd doesn’t exist otherwise it keep on appending “Param” xml node to “Params” root:


public static void WriteToFile(string paramValue, string paramType)
{
const string FILE_NAME_WITH_PATH = @"RecordedParams.xml";
const string ROOT_NODE_NAME = "Params";
const string CHILD_NODE_NAME = "Param";
const string PARAM_TYPE_ATTRIBUTE_NAME = "ParamType";
XDocument xDocument = null;
if (File.Exists(FILE_NAME_WITH_PATH)){
xDocument = XDocument.Load(FILE_NAME_WITH_PATH);
}

if(xDocument == null){
xDocument = new XDocument(
new XDeclaration("1.0", "UTF-8", "yes"),
new XElement(ROOT_NODE_NAME, new XElement(CHILD_NODE_NAME,
paramValue,
new XAttribute(PARAM_TYPE_ATTRIBUTE_NAME, paramType)))
);
}
else{
var paramS = xDocument.Element(ROOT_NODE_NAME);
paramS.Add(new XElement(CHILD_NODE_NAME, paramValue,
new XAttribute(PARAM_TYPE_ATTRIBUTE_NAME, paramType)));

}

xDocument.Save(FILE_NAME_WITH_PATH);
}

The xml looks like this. Note the Param value can be JSON represesntation of any object:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Params>
<Param ParamType="TypeA">
[{"Key":"city","Value":"Los Angeles"},{"Key":"state","Value":"CA"}]
</Param>
<Param ParamType="TypeB">
[{"Key":"city","Value":"Atlanta"},{"Key":"state","Value":"GA"}]
</Param>
</Params>

2. Access the xml file in your test.

3. De-serialize and Use.

So here I just create a simpley MS Unit test with the XML as deployment item, so I can access it from Linq  to XML (note you can just have a JSON file instead too).

The test will like this:

[TestMethod, DeploymentItem ("RecordedParameters/RecordedParams.xml")]

public void Get_Params_And_Call_A_Service()

{

Dictionary<string, string> parameters = null;

 

string recordedParamPath = "RecordedParams.xml";

 

XDocument xDocument = XDocument.Load(recordedParamPath);

var para =

xDocument.Element("Params").Elements("Param")

.Where(e => (e.Attribute("ParamType") !=null

&& e.Attribute("ParamType").Value == "YourAttributeName") );

 

var firstParam = para.Last().Value;

 

parameters = firstParam.FromJson<Dictionary<string, string>>();

There is one important thing you will have to note here, make sure that you right click on the file and go to its properties and choose Copy Always from “Copy to Output Directory” option.

Without doing this you will get IO exceptions.

Also note that I have an extension method for string to serialize it to an object.  It is a very simple function:

public static T FromJson<T>(this string json)

{

var returnObject = Activator.CreateInstance<T>();

var serializer = new DataContractJsonSerializer(returnObject.GetType());

 

using (var stream = new MemoryStream(Encoding.Default.GetBytes(json)))

{

return (T)serializer.ReadObject(stream);

}

}

You can also use DynamicJson if you are using 4.0 framework.

 

Happy testing!

 

Is it time to go from XML to JSON?

 The answer is a qualified yes. Qualified because, of course, there is a place for XML; however, I think it is time for an exodus towards JSON.  More specifically, we need this exodus for storing simple data files for settings, etc, in a web application.

My first argument for doing so is perhaps common sense, but nevertheless important to mention: we are using JSON everywhere in JavaScript for numerous reasons, one being most modern browsers support native parsing of it, so it is fast and robust. Therefore, for consistency sake why can’t use JSON on the server side too?

Consistency argument only goes so far if a language doesn’t support the paradigm/ technology very well. Hence, my second argument:  asp .net 4.0 framework now supports dynamic variable which opens up the gate for true JSON implementation on server side.

Don’t worry you don’t need to do any work. Somebody else already have an elegant library here:

http://dynamicjson.codeplex.com/

So how would the implementation look like? Using the above library now you can parse simple JSON text file, and use it as following:
Let’s say your JSON string was following:

{
"TrafficLights":{
    "GREEN" : "Your OK to pass",
    "RED" : "Stop",
    "YELLOW" : "Accelerate",
}
}

Then you can access it using DynamiJson:
              Snippet

 var result = DynamicJson.Parse(jsonString);

                if (result != null)
                {

                    var TrafficLights = result.TrafficLights;

                    Response.Write(TrafficLights.GREEN);

                }

In Client Side (Javascript) you will have something like this (using jQuery):

$.getJSON('/yourjsonfilepath/trafficlights.json.txt', function (data) {
TrafficLights = data. TrafficLights;
});

alert(TrafficLights.GREEN);

Looks easy enough? I think so, and how about cleanliness and consistency. I think it is pretty clean too. Also, this can give you performance juice, if you are not loading all JSON files on page load, but instead doing it through a delayed AJAX call.  So start your exodus now.

 

Refactoring Code with Complexity Analysis: Visual Studio Code Analysis and Source Monitor

Remember the time when you had to start working on a new project, where you had to look into this jumble of spaghetti that somebody called code?

No? Well, I do and let me tell you it is not a pretty picture.

I wanted to refactor, and disentangle the mess, but frankly I didn’t know where to start.  It was just a struggle to get started.

Recently, I realized that ,perhaps, I could have used a complexity analysis tool to find out most complex methods, classes and start from there. In fact, I was just introduced to a tool called source monitor.

It is a powerful tool that gives you simple complexity averages with Kiviat Graph.

 

 

 

 

 

 

 

 

Soon after I realized that Visual Studio have a built in code analysis tool too. To get some metrics, all you have to do is really go to “Analyze” menu and choose code analysis:

 

 

 

Now you have a list of metrics, and most importantly  a maintainability index.  This index is a number in the range 0-100. Zero being the worst and 100 the best. This maintainability index according msdn is :

Maintainability Index = MAX(0,(171 – 5.2 * ln(H alstead Volume) – 0.23 * (Cyclomatic Complexity) – 16.2 * ln(Lines of Code))*100 / 171)

True, above formula can induce nausea, but all you have to know is that it is a function of complexity and lines of code.

 

Here you can see it in action, highlighting a really bad method that have score of 2 for maintainability:

 

If we look at the method highlighted above you will see that I have put a lot of if statments and loops:

 

                   for (int i = 1; i < 100; i++) c = a;
            if (a == 1)
                if (a == 3)
                    if (b == 1)
                        if (c == 2)
                            for (int i = 1; i < 100; i++) c = a;
            for (int i = 1; i < 100; i++) c = a;
            for (int i = 1; i < 100; i++) c = a;
            for (int i = 1; i < 100; i++) c = a; for (int i = 1; i < 100; i++) c = a; for (int i = 1; i < 100; i++) c = a;

 

Definitely not maintainable? Indeed! Visual Studio did its job and this index seems to be useful. In fact, we can use it as a criterion for refactoring.  You can find the least maintainable methods and create work items, and start refactoring.

Creating work items is integrated with analysis window, simply right click on the method and choose create work item:

 

 

 

I think using code complexity gives a structure and most importantly a starting point for refactoring. Try it in your code base, and get a feel for it yourself. Visual Studio code analysis is definitely more easier to use, but not as advance as source-monitor.

Dynamic Data, oData, and WCF Data Services a Killer Combination

Yes the title is lame. Yes it is a plea for your attention. Now that we have this out of the way, let’s move on.

I will attempt to build a web application for GRE Preparation to demonstrate how Dynamic Data, oData, WCF Data Services with jQuery can be used to build, very rapidly, a very powerful web application.

Let’s dive into the meat of this right away. From initial analysis, I realize that I need to do following four things to get started:

 

  1. I need to define my data objects first
  2. I need an admin page where I can enter and update data
  3. I need a service layer that can expose my data
  4. I need my front end to consume the data from services

 

 

 

1.    Define your Data Object with Entity Framework

Let’s focus on point one first. We need to define our data objects. Remember we are not talking about a database yet. All we need to figure out is data objects. We have perfect tool to use here, entity framework. So right click on your project and choose “add new item”.

 

 

 

Then select ADO.NET Entity Data Model. After you added the model you will get this empty canvas where you can right click and add entities.

By repeating this process several times and then adding some associations I ended up with something like this for my project:

 

 

Time for some magic. Right Click on the model and click generate database:

 

 

Now you can take the generated script and execute it against your Database System. Abracadabra you have your database now!

2. Create Your Admin Page with Dynamic Data

Now that I have my database, I need a way to enter data. Sure I can enter data using SQL Management Studio, or Visual Studio, but I would eventually need an application to let my non geeky fellows enter da ta. Also, I need something user friendly.  What I need is to expose all my tables through a web app. I need to use Dynamic Data. I simply add new project and choose Dynamic data with Entity Framwork.

 

 

 

Believe it or not, one more change  and we are done. Go to Global.asax, and choose your EntityFramework.

 

 

DefaultModel.RegisterContext(typeof(YourDataContextType), new ContextConfiguration() { ScaffoldAllTables = false });

 

Change YourDataContextType to your entity class in my case GREPrepDBEntities. Rebuild and you are done. You have your admin. Phew!

 

 

 

 

 

 

 

I know what you are thinking: you are thinking that all this automatic code generation will be really hard to understand and modify, so you will end up spending way more time optimizing and customizing.  Good news is dynamic data is highly customizable, and the project give you access to all of generated code. You can modify pretty much everything. More on that later! For now we have created as simple applications that give us CRUD operations out of the box.

 

3. Create Your Service Layer with WCF Data Services

 

We have our data, we have created a way to enter and update it too. Next we focus on providing this data openly to any application, a front end web application, iPhone application, etc.

Here is a short list of features we need from the service:

 

  1. Rest full calls over HTTP
  2. Ability to return JSON for our AJAX calls
  3. JSONP calls for cross domain AJAX calls

 

These are just few features we can really use in our application. Everything else would be a definite bonus. That bonus could include a standard way of querying data such as oData.

WCF Data services provide, out of the box, oData support which will give you the ability to query update and delete your data. Also it has the ability to return data in XML and JSON. Furthermore, we can easily extend it to provide JSONP.

Without further ado we get started.  Add new item in your project and choose WCF Data Services.

 

 

After adding the data service, just like Dynamic Data, we now configure it to use our entity model:

 

public class GREPrepDataService : DataService< GREPrepDBEntities>
Next we configure our Data Service to allow all Tables to be exposed:
 // This method is called only once to initialize service-wide policies.
        public static void InitializeService(DataServiceConfiguration config)
        {
            // TODO: set rules to indicate which entity sets and service operations are visible, updatable, etc.
            // Examples:
             config.SetEntitySetAccessRule("*", EntitySetRights.All); 

Thats it! We have our powerful service configured.

4. Front End Web App with jQuery Template

Here comes the most exciting part. Building the consumer facing application is always the most thrilling. As our first task we consider simply outputting our data in a html. We create a simple web application and then we use recently proposed Microsoft jQuery template engine to output JSON data from our service. We first define our html and template:

   <script type="text/html" id="contactTemplate">
        <div>
            <h2>{{= ConceptName}}</h2>
            <br />
            <p>{{= ConceptDescription}}</p>           �
        </div>
   </script>
<div id="math-core"></div>

Note the script type is “text/html”. If you don’t have this, it will not work. This template as I mentioned before is part of Microsoft’s proposal for jQuery. You read all about it in Scott Guthrie’s post here.

Next we make our jQuery AJAX  call to get data:

$.ajax({
        dataType: "json",
        url: query,
        success: callback
    });

In the callback we simply bind the data with the template:

function callback(result) {  �
        var mathcore = result["d"]["results"];
        var t = $("#contactTemplate").render(mathcore).appendTo("#math-core");

    }

This is it. This will produce following html from the returned data based on the jQuery template we defined:

<div>
        <h2>
            Integers</h2>
        <br>
        <p>
            An integer is all counting numbers</p>
    </div>
    <div>
        <h2>
            Even Integers</h2>
        <br>
        <p>
            Integers that are multiple of 2 are called even integers</p>
    </div>

Our page will look like this:
 
To make our service work with JSONP we just have to download the extension from here, and the attribute to your service:

[DataServicesJSONP.JSONPSupportBehavior]
    public class GREPrepDataService : DataService< GREPrepDBEntities>
    {
 

After that we can simply change our jQuery call to use JSONP callback, and we get the same result except now you can make cross domain calls:

 
function callback(result) {
        // unwrap result         �
        var mathcore = result["d"]["results"];
        var t = $("#contactTemplate").render(mathcore).appendTo("#math-core");

    }

    // Build OData query     �
    var query = "http://localhost:7724/GREPrepDataService.svc/MathCores" //�
             + "?$callback=callback&$expand=MathExamples" // jsonp request  

    // Make JSONP call
    $.ajax({
        dataType: "jsonp",
        url: query,
        jsonpCallback: "callback",
        success: callback
    });

This is it for now. We used Dynamic Data to create an admin application, and WCF Data Services to provide a powerful restful API which is in turn consumed by
a front end that used jQuery template proposed by Microsoft. Now you have an example of using these new technologies together. Hopefully, it was helpful for you.

 

 

Google Analytics plugin by http://pokercalculator-free.com