Interface level validations of DataObjects

Purpose

I was recently working on a project where I wanted to create the most granular interfaces possible and then group them into usable objects. My intent was to pass these objects around to shared utilities and services that would focus on the interfaces and thus be reusable by intent instead of specific implementation. For example, we would create an interface each for phone number, email and street address. These interfaces could be used on a customer, vendor, or employee object to ensure a standard implementation of the fields. Then, we could build our communications services to work off the interfaces for communication types and thus be generic to the entity being contacted. By doing this, I would be able to build singularly focused services that dealt with small tasks on data objects with a disregard for their specific implementation beyond the interface.

Validation

Being a big fan of data contracts with ability to run self-aware validations(e.g. validations that can only consider the scope of the contract itself without any relationships), I found myself needing a way to have all objects validate based on the requirements of the smaller interfaces, within the CRUD services of the manifested object. I also wanted to ensure the ability to dynamically add validation routines and have them picked up by the calling services without having to couple them together.

Solution

By creating a custom attribute and applying it to methods in the interface, I would be able to reflect through the methods of any object and trigger it to self validate using any and all methods that were added to the object from the implemented interfaces. Next, we simply call a data contract extension method that to invoke all validations. Here are the examples.
Custom Method Atribute

[AttributeUsage(AttributeTargets.Method, Inherited = false)]
public class InterfaceValidation : Attribute
{
    //...
}

Data Contract Extension

public static class DataContractExtensions
{
    public static T CallInterfaceValidations<T>(this T obj)
    {
        var methods = obj.GetType().GetMethods();
        object[] parameters = null;
        foreach (var method in methods)
        {
            var attributes = method.GetCustomAttributes(typeof(InterfaceValidation), true);
            if (attributes != null && attributes.Length > 0)
            {
                try
                {
                    method.Invoke(obj, parameters);
                }
                catch (Exception)
                {

                    throw;
                }

            }
        }
        return obj;
    }
}

Assign the attribute to a method

[InterfaceValidation]
void ValidateSomething();

Inversion of Control Pattern

Dependency Inversion Principle

The DIP states that high level classes should not depend on low level classes.  Both should depend on abstractions.  Details should depend on abstractions.  Abstractions should not depend on details.  The DIP is about reversing the direction of dependencies from higher level components to lower level components such that lower level components are dependent upon only the interfaces owned by the higher level components.  This is a method for moving to a more loosely coupled architecture.  Basically, you have to depend on a standard interface used by objects and not depend on their details.  This one might be best illustrated by example. 

In the following example, the BadCar class is tightly coupled to the actual implementation of the BadMotor function.  This means that these two objects are married, and changes to one directly impact the other.

public class BadMotor

{

    public Boolean Start()

    {

        Console.Write("Starting");

        return true;

    }

}

//tightly coupled to details.

public class BadCar

{

    public BadMotor Motor {get;set;}

    public Boolean Start(BadMotor badMotor)

    {

        Motor = badMotor;

        return Motor.Start();

    }

}

While this does function, it creates a dependency relationship between two objects at their implementation level.   This creates hardships for maintenance and scalability long term.   The proper way to build this relationship would be to invert dependencies onto interfaces to ensure no object has knowledge or visibility into implementation of any other object. Consider the following examples:

public interface IEngine

{

    bool Start();

}

public interface IGreenEngine : IEngine

{

    bool IsCharged ();

}

public class FourCylEngine : IEngine

{

    public bool Start()

    {

        Console.WriteLine("4Cyl Starting");

        return true;

    }

}

public class V8Engine : IEngine

{

    public bool Start()

    {

        Console.WriteLine("V8 Starting");

        return true;

    }

}

public class HybridEngine : IGreenEngine

{

    public bool Start()

    {

        if (IsCharged())

            Console.WriteLine("Hybrid Starting");

        return true;

    }

    public bool  IsCharged()

    {

        Console.WriteLine("Hybrid is charged");

        return true;

    }

}

public class Car

{

    public Car(IEngine engine)

    {

        Engine = engine;

    }

    public IEngine Engine{get ; set ; }

    public Boolean Start()

    {

        return Engine.Start();

    }

}

I built this interface dependency and inheritance to ensure that objects are loosely coupled and only share an interface.   This allows for actual base implementations to come and go and even be recognized dynamically without causing lower level objects to update their implementation.   Here is an example of hot swapping and scaling with the previously defined objects and interfaces.

class Program

{

    static void Main(string[] args)

    {

        //  Cars are only dependant upon an Engine interface

        Car BigThing = new Car(new V8Engine());

        BigThing.Start();

        //  Cars are only dependant upon an Engine interface

        Car SmallThing = new Car(new FourCylEngine());

        SmallThing.Start();

        // Since we have an interface dependency, it is easy to hot swap.

        BigThing.Engine = new HybridEngine();

        BigThing.Start();

    }

}

The output is as follows:

As you can see, this allows for maintainability and long-term scalability by ensuring that the objects stay out of each other's business. By adhering to the spirit of this principle as well as the previous SOLID principles, you can keep your code base healthy and easy to maintain when the requirement changes come.

OData , Atom and AtomPub

The Open Data Protocol (OData) is a protocol which standardizes the exposure and consumption of data. In times where data is being exposed at high rates and where consumers connect to more and more data endpoints, it’s important for clients to access these endpoints in a common way. OData builds on standards like HTTP, Atom, and JSON to provide REST access to controller based endpoints. Data is exposed as entities where each entity can be treated as an Http resource which makes it subject to CRUD (create, read, update, delete) and Patch operations.

Atom is way to expose feeds much the same way RSS does. Atom by itself allows only feed exposure. If you want to publish data, AtomPub (Atom publishing) provides this ability. AtomPub uses HTTP verbs GET, POST, PUT, and DELETE to enable data publishing.

This is not an implementation to be used in every situation obviously.  But it is an interesting flavor of having feed available data with real interaction.

Team Foundation Services is now Visual Studio Online

Microsoft officially launched Visual Studio Online (Formerly Team Foundation Services) last week.  The announcement came with news of many added features and benefits.  Here is an overview of the announcement:

Pricing

The good news is that if you have MSDN, you are most likely going to still not be charged for day-to-day usage. The bad news is that if you have product owner or product manager roles our your team and want them to use VSO for backlog interaction, etc.  They will now need to pay a membership fee.  The MSDN license that allows that access free, is the Ultimate level, which is the most expensive MSDN package there is.   This is going to cost at least $45 monthly for these employees (of course depending upon their role in your organization).  You can find the breakdown of pricing levels here.  The other financial consideration is that you will be charged for build time on any build and deploy jobs.  This is basically going to cost a couple of pennies per minute every time you do a CI style build. I did some investigation and found out that this does include publish time. In our world, that is roughly 65% of the time the build job is running.  So, you will pay the going rate while your build is being uploaded and deployed to the Azure instances for which you are already paying.

Link VSO with Azure

Link your VSO account to your Azure account and have a single portal for managing them both.  Very handy.  Details here.

Monaco

Monaco is a new development service specifically designed for building and maintaining Windows Azure Websites. With Monaco, developers have a lightweight free companion to the Visual Studio desktop IDE that is accessible from any device on any platform. Monaco is a rich, browser based, code focused development environment optimized for the Windows Azure platform, making it easy to start building and maintaining applications for the cloud.  Here are some cool videos on Channel9

Application Insights

With this “360 degree view” of your application, Application Insights can quickly detect availability and performance problems, alert you, pinpoint their root cause and connect you to rich diagnostic experiences in Visual Studio for diagnosis and repair. It also supports continuous, data-driven improvement of an application. For example it highlights which features are most and least used, where users get “stuck” in an application, where and why exceptions are occurring, which client platforms are being used with which OS versions, and where performance optimizations will make the biggest impact on compute costs.  You can sign up for the free preview here.  Following are some sample screen shots:

Dashboard

Visual Studio Integration

Environment Metrics

Quick method to optimize your foreign key searching

A lot of people do not realize that creating a foreign key does not also create an index.  This is by design and actually a good thing.  Over indexing a table can actually slow querying as every insert or update causes indices to be recalculated.  Over indexing a table will also slow down select statements from that table as the query optimizer will struggle through evaluating all of the indices to pick the one it thinks is best suited for your current search. 

When working on building targeted foreign key indices to speed up a search, I came up with this code block to auto generate the script for me.

SELECT 'IF NOT EXISTS (SELECT * FROM sys.indexes WHERE name = N''IX_' + FK.TABLE_NAME + '_' + CU.COLUMN_NAME + ''') 
BEGIN 
    CREATE INDEX IX_' + FK.TABLE_NAME + '_' + CU.COLUMN_NAME + '  ON ' + FK.TABLE_NAME + '(' + CU.COLUMN_NAME + ');
END
GO
print N''create IX_' + FK.TABLE_NAME + '_' + CU.COLUMN_NAME + ' done''; 
RAISERROR (N'' --------------------'', 10,1) WITH NOWAIT',
       FK.TABLE_NAME AS K_Table,
       CU.COLUMN_NAME AS FK_Column,
       PK.TABLE_NAME AS PK_Table,
       PT.COLUMN_NAME AS PK_Column,
       C.CONSTRAINT_NAME AS Constraint_Name
FROM   INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS AS C
       INNER JOIN
       INFORMATION_SCHEMA.TABLE_CONSTRAINTS AS FK
       ON C.CONSTRAINT_NAME = FK.CONSTRAINT_NAME
       INNER JOIN
       INFORMATION_SCHEMA.TABLE_CONSTRAINTS AS PK
       ON C.UNIQUE_CONSTRAINT_NAME = PK.CONSTRAINT_NAME
       INNER JOIN
       INFORMATION_SCHEMA.KEY_COLUMN_USAGE AS CU
       ON C.CONSTRAINT_NAME = CU.CONSTRAINT_NAME
       INNER JOIN
       (SELECT i1.TABLE_NAME,
               i2.COLUMN_NAME
        FROM   INFORMATION_SCHEMA.TABLE_CONSTRAINTS AS i1
               INNER JOIN
               INFORMATION_SCHEMA.KEY_COLUMN_USAGE AS i2
               ON i1.CONSTRAINT_NAME = i2.CONSTRAINT_NAME
        WHERE  i1.CONSTRAINT_TYPE = 'PRIMARY KEY') AS PT
       ON PT.TABLE_NAME = PK.TABLE_NAME
WHERE  PT.Column_name = 'ID'
       AND PK.Table_Name = '{COMMONLY QUERIED TABLE}'
       AND FK.TABLE_NAME <> PK.TABLE_NAME;  

In this statement, I would replace {COMMONLY QUERIED TABLE} with the table name holding the primary key that was used in queries often. This will generate statements to create an index if one does not exist. You could easily modify it to do a drop and add as well.

ASP.NET Identity for 4.5

ASP.NET membership has gone through many changes over the years. From simple membership to SQLProviders to OWIN, the needs of developers are constantly changing. .Net 4.5 has brought another change to the identity model. We have to let go of the assumption that users will log in by entering unique credentials to our application. Increasingly, users expect to leverage a single online identity to drive all of their web-based experiences (e.g. Facebook, Twitter, etc.) Developers should also want users to be able to log in with these social identities so that our applications can provide a rich and integrated experience to the users' online life.

Unit testing code should be a core concern for application developers. MVC is a great pattern and platform for those who want to unit test their code.  Now, you should easily be able to do that with the membership system. ASP.NET Identity was developed with the following goals (Verbatim from Microsoft):

• One ASP.NET Identity system 
• ASP.NET Identity can be used with all of the ASP.NET frameworks, such as ASP.NET MVC, Web Forms, Web Pages, Web API, and SignalR. 
• ASP.NET Identity can be used when you are building web, phone, store, or hybrid applications.
•  Ease of plugging in profile data about the user 
• You have control over the schema of user and profile information. For example, you can easily enable the system to store birth dates entered by users when they register an account in your application. 
•  Persistence control 
• By default, the ASP.NET Identity system stores all the user information in a database. ASP.NET Identity uses Entity Framework Code First to implement all of its persistence mechanism. 
• Since you control the database schema, common tasks such as changing table names or changing the data type of primary keys is simple to do. 
• It's easy to plug in different storage mechanisms such as SharePoint, Windows Azure Storage Table Service, NoSQL databases, etc., without having to throw System.NotImplementedExceptions exceptions. 
• Unit testability 
• ASP.NET Identity makes the web application more unit testable. You can write unit tests for the parts of your application that use ASP.NET Identity. 
• Role provider 
•  There is a role provider which lets you restrict access to parts of your application by roles. You can easily create roles such as “Admin” and add users to roles. 
• Claims Based 
• ASP.NET Identity supports claims-based authentication, where the user’s identity is represented as a set of claims. Claims allow developers to be a lot more expressive in describing a user’s identity than roles allow. Whereas role membership is just a boolean (member or non-member), a claim can include rich information about the user’s identity and membership. 
• Social Login Providers 
• You can easily add social log-ins such as Microsoft Account, Facebook, Twitter, Google, and others to your application, and store the user-specific data in your application. 
•  Windows Azure Active Directory 
• You can also add log-in functionality using Windows Azure Active Directory, and store the user-specific data in your application. For more information, see Organizational Accounts in Creating ASP.NET Web Projects in Visual Studio 2013 
• OWIN Integration 
• ASP.NET authentication is now based on OWIN middleware that can be used on any OWIN-based host. ASP.NET Identity does not have any dependency on System.Web. It is a fully compliant OWIN framework and can be used in any OWIN hosted application.
• ASP.NET Identity uses OWIN Authentication for log-in/log-out of users in the web site. This means that instead of using FormsAuthentication to generate the cookie, the application uses OWIN CookieAuthentication to do that. 
• NuGet package 
• ASP.NET Identity is redistributed as a NuGet package which is installed in the ASP.NET MVC, Web Forms and Web API templates that ship with Visual Studio 2013. You can download this NuGet package from the NuGet gallery. 
• Releasing ASP.NET Identity as a NuGet package makes it easier for the ASP.NET team to iterate on new features and bug fixes, and deliver these to developers in an agile manner.

ASP.NET SignalR

ASP.NET SignalR is a library that allows developers to add real-time web functionality to applications quickly and easily. The library wraps various techniques and creates a unified toolkit for server to client push notifications. SignalR will leverage the best technique for bi-directional communication available for the current browser and server by first attempting to connect via WebSockets and then falling through to long polling. The best part is that your application code stays the same no matter what communication methodology is in use. SignalR also provides a very simple, high-level API for doing server to client RPC (call JavaScript functions in your clients' browsers from server-side .NET code) in your ASP.NET application, as well as adding useful hooks for connection management, e.g. connect/disconnect events, grouping connections, authorization.

SignalR can be used to add any sort of "real-time" web functionality to your ASP.NET application. While chat is often used as an example, you can do a whole lot more. Any time a user refreshes a web page to see new data, or the page implements Ajax long polling to retrieve new data, is candidate for using SignalR. It also enables completely new types of applications, that require high frequency updates from the server, e.g. real-time gaming. For a great example of this, see the ShootR game

Get it on NuGet by running:

Install-Package Microsoft.AspNet.SignalR

Get a sample on NuGet, straight into your app by running:

Install-Package Microsoft.AspNet.SignalR.Sample

Securing a Web API with Windows Azure AD and Katana

I just read through this article doing a walk-through on how to secure your WebAPI with Azure AD using Katana and OWIN.  It lays out all of the benefits of of OWIN and using Katana. 

http://www.cloudidentity.com/blog/2013/07/23/securing-a-web-api-with-windows-azure-ad-and-katana/

Worth the read.

Windows Azure Notification Hubs

Windows Azure Notification Hubs was just released for general availability.  These hubs help mobile app developers deliver large numbers of push notifications to mobile users on a wide range of platform.Windows Azure Notification Hubs makes sending push notifications through multiple notification services achievable with just a few lines of code.  Today, apps like the Bing News for Windows 8 are using notification to send millions of push notifications to inform users of the latest breaking news. 

This doesn't replace Windows Azure Mobile Services, which  also supports push notifications. The table below explains the differences.   In short, Mobile Services is best used for communicating to a single user whereas Notification Hubs is best used for communicating to many users simultaneously.

	Mobile Services	Notification Hubs
MPNS, WNS, APNS, and GCM support	Yes	Yes
Turnkey event-triggered push	Yes	No
Device registration management	No	Yes
Interest tags for routing messages to a subset of users	No	Yes
Templates for formatting messages to user preferences including language	No	Yes
Broadcast to >1 million devices at once within minutes	No	Yes

Check out Scott Gu's post for more details.

OWIN and the Katana Project

OWIN

OWIN (a.k.a. Open Web Interface for .NET) is a new way of processing HTTP requests in .NET.  It is a specification describing how .NET web servers and .NET applications should interface and interact. The idea is that by decoupling the server and application we are able to write code independent of the host platform.  Our code at that point can be host agnostic and allow us much more flexibility on our chosen publishing environment.

IIS currently does double duty by handling both the host and server responsibilities.  To be clear, by host duties, I mean the process management, and by server, I mean network management and request handling.  This means that we as developers have to use packages specific to IIS suck as HttpModules, HttpHandlers, etc.  While this is effective, it makes our application tightly coupled to a specific hosting environment.  So, if we would want to target something a little more small to host a WebAPI, we would be out of luck.  OWIN gives us the ability to decouple those two things which gives us the flexibility to choose components based on individual functions while selecting only the ones we need and substituting only the ones that are actually different across hosting platforms. All of those changes would actually take place beneath the application.  Hm.. where have I seen this kind of thing before?

The way I understand it, a server in OWN is made up of an environment dictionary, of the form IDictionary<string, object>, used for holding the request processing state: things like the request path, the headers collection, the request bits themselves, and a delegate, of the form Func<IDictionary<string, object>, Task>, which is used to model how the components appear to each other.  This means that an app is a list of those components executed in sequence, each of those awaiting the next and passing the environment dictionary to each other.  This seems very dynamic and portable, which is a really strong position for a WebAPI or other hosted service. 

Katana Project

Katana is a set of components by Microsoft for building and hosting OWIN-based web applications.  Katana has host, server, and middleware source code and documentation located here. These tools are available via NuGet.  These products are actively developed by the Katana team assigned to the Microsoft Open Tech Hub and in collaboration with a community of open source developers.

ASP.NET MVC4 - Mobile Views

In ASP.NET MVC4 you have the ability to easily present a view directed at a mobile display without requiring the use of an add on tools.  Sure, for a good mobile experience, you may choose to implement jQueryMobile or something similar, but the fact remains that it is not required with this functionality.

Mobile Experience

A good portion of visual design for mobile can come from using simple media queries to detect the current size of the users' device and render a visual experience accordingly.   Media queries within CSS are straightforward and can handle the majority of simple user experience changes.  For instance, if you wanted to change any properties of an element based on the screen width and orientation of the users' device, you could simply create some combination of the following code:

/* #### Mobile Phones Portrait #### */
@media screen and (max-device-width: 480px) and (orientation: portrait){
  /* some CSS here */
}

/* #### Mobile Phones Landscape #### */
@media screen and (max-device-width: 640px) and (orientation: landscape){
  /* some CSS here */
}

/* #### Mobile Phones Portrait or Landscape #### */
@media screen and (max-device-width: 640px){
  /* some CSS here */
}

/* #### iPhone 4+ Portrait or Landscape #### */
@media screen and (max-device-width: 480px) and (-webkit-min-device-pixel-ratio: 2){
  /* some CSS here */
}

/* #### Tablets Portrait or Landscape #### */
@media screen and (min-device-width: 768px) and (max-device-width: 1024px){
  /* some CSS here */
}

/* #### Desktops #### */
@media screen and (min-width: 1024px){
  /* some CSS here */
}

That sort of thing will go a long way, but sometimes it can lead to clutter and hard to follow CSS.

Overriding a mobile view in MVC4

With MVC 4, there is a very simple mechanism that lets you override any view for mobile browsers in general.  You can also define your own parameters for a specific mobile override, giving you the ability to build specific views for specific devices or user agents.  To provide a view that overrides for any mobile device, all you have to do is copy the view to a new file and add .Mobile to the file name. For example, to create a mobile Something view, copy Views\Home\Something.cshtml to Views\Home\Something.Mobile.cshtml.  The current context will trigger which view is pulled in a run time for a given client.  Now, you have effectively created a mobile specific view that can be visually altered for an experience geared at a mobile platform.

Custom Override Views

You can create almost any kind of custom mobile view you would like by defining a context condition and inserting into the DisplayModeProvider instance. For example, the following code will create a specific display mode based on check the user agent of the current context against 'iPhone'. Then, you add it to the list of display modes within the ApplicationStart method of the global.asax:

public class MvcApplication : System.Web.HttpApplication 
 {
 protected void Application_Start()
    {
    AreaRegistration.RegisterAllAreas();

     DisplayModeProvider.Instance.Modes.Insert(0, new DefaultDisplayMode("iPhone")
        {
         ContextCondition = (ctx =>
         ctx.Request.UserAgent.IndexOf("iPhone", StringComparison.OrdinalIgnoreCase) >= 0)
        });

    //Add another one specifically for some tablets too 
    DisplayModeProvider.Instance.Modes.Insert(0, new DefaultDisplayMode("Tablet")
        {
         ContextCondition = (ctx =>
         ctx.Request.UserAgent.IndexOf("iPad", StringComparison.OrdinalIgnoreCase) >= 0 ||
         ctx.Request.UserAgent.IndexOf("Android", StringComparison.OrdinalIgnoreCase) >= 0 &&
         ctx.Request.UserAgent.IndexOf("Mobile", StringComparison.OrdinalIgnoreCase) <= 0
        )
        });
    FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
    RouteConfig.RegisterRoutes(RouteTable.Routes);
    BundleConfig.RegisterBundles(BundleTable.Bundles);
    }
 }

As you can see, the framework gives you the ability to create as many focused views, with as much device granularity as you see fit for your application. This will help ensure good user experiences on specific devices without having to rework existing CSS or get into switching to mobile sites.

ASP.NET Web API

The ASP.NET Web API framework was introduced recently into ASP.NET.  I touched on it some previously at an overview level, but would like to dig in a little more at this time to show the power of having a true Web API in your product.  

Overview

From a high level, leveraging the Web API framework in ASP.NET allows you to easily build HTTP based services geared at reaching browsers, desktop or mobile clients.  By creating a responsive layer of RESTful APIs for a product, the platform immediately becomes a much more adaptable and scalable solution.  The simple act of ensuring all 'real' application logic is handled behind a RESTful API call, an inherent separation of interests is manifested, which enables a more loosely couple application at every layer.

ASP.NET Web API Infrastructure

At its base, the ASP.NET Web API mechanism is a simple adaptation of an HTTP service.  Web API can be used along with WCF, but realistically, they are two means to create virtually the same thing.  The main aspect of a Web API layer is the ability to create an API project within a given solution that allows user stories to be executed via RESTful calls to the services.  Microsoft has a great tutorial for creating a simple Web API project as an introduction to the platform.  For those familiar with MVC, this exercise should be very straightforward and easily worked through.  In fact, as you work through this, it will probably become clear that there is very little difference here than a standard MVC application.  The main difference is that Web API uses the HTTP method, not the URI path, to select the action. But, you can also use MVC-style routing in Web API if that is more your style.

Dynamism

Leveraging an API infrastructure as a service layer for you web application gives you the ability to manifest any client on any platform without recreating application intelligence. Some of the main features of Web API allow for the creation of a dynamic service layer that can interact with multiple versions of the same platform, 3rd party vendors, customer integrations and most importantly, your own application.  A strong API will have an entry point to accommodate the same list of user stories that the main presentation layer in your application presents to a user.  The API layer should handle modeling, security, extensibility and scalability.  This allows for the consistent use of application logic and business rules across multiple consuming platforms.  API/SDK layers should be manifested by version, with all historic versions co-existing.  A single set of business logic and data access logic should persist across versions of API.  This allows forward movement without backward degradation.  By adhering to the rule of the eternal public interface, this is an achievable goal.

The 'Real' API

While the main focus of this post is the ASP.NET Web API offering, I would like to make a point regarding what an API should be in my opinion.  An API should be a complete manifestation of every user story that exists in your application.  The API should be a service layer that is not the data access layer, nor is it the main business logic layer.  It is the manifestation of User Stories.  This type of layer has been called the User Story Layer and the services contained within have been called Complex Services.  This is a logical regrouping, or coupling of specific business logic tasks to create a user story.  A user story should be completely executed within its single methodological representation, including input validation, security and access validation, activity logging and return values. All manifested user story methods should assume no knowledge of requirements, modalities, system or data flow from any client executing the method. Data access and business logic should never be directly exposed to an end user, third party or otherwise.  Data access and business logic should be separated from each other as well as from the API/SDK/USL.  This enables the ability to quickly couple existing sets of logic into a new user story without having to accommodate situational elements inside the business logic layer.  For more information on building a strong SOA, see some of my previous posts here.  For a theoretical discussion of the purpose behind SOA, check this post.  A framework that has been built with this decoupled approach of singularly responsible object residing behind globally accessible services will much more easily allow for productivity items such as Continual Integration, Continual Deployment, and fully automated testing.

Your First API Client

The short answer here is that it’s your application.  A robust API should first and foremost serve all logic to your own applications.  You have to create your API and then consume it.  This ensures that all changes made are not only accounted for in the API, but are placed there with intent.  You have to eat your own dog food.  In a web development and agile deployment environment, the ability to decouple code and responsibility of objects is the best friend you will ever have.  Approach your web design with intent and long term vision for things like scalability and performance.  Understand that you need to support multiple versions of your service layer and extend your data objects over multiple versions.  A properly architected system will ensure you are able to respond to changes in your market and your market requirements much more quickly than would be the case otherwise.

Absence

I have had a hectic couple of weeks and have not been able to focus much time on blogging. I will be back at it soon.

C# DateTime to Javascript issues with timezone

I ran into an interesting problem when serializing a C# DateTime object as JSON and returning it via a controller to javascript.  The timezone mechanism was causing my times to shift to a localized time based on the originating timezone offset to the local timezone of the web host.  Normally, that might not be a huge issue, but since my application was displaying appointments which were being pulled from a localized data store to a user in that same region, the server timezone is irrelevant and the time for the appointment has to be the same time regardless of the timezone.  After much digging around, I found this little snippet that seems to answer the problem.

function parseJsonDate(jsonDate) {
    var offset = new Date().getTimezoneOffset() * 60000;
    var parts = /\/Date\((-?\d+)([+-]\d{2})?(\d{2})?.*/.exec(jsonDate);

    if (parts[2] == undefined) parts[2] = 0;
    if (parts[3] == undefined) parts[3] = 0;

    return new Date(+parts[1] + offset + parts[2] * 3600000 + parts[3] * 60000);
};

IIS Issue - Ignoring default document

We were having some issues trying to get an Azure app to show the proper page by default when hitting the root domain.  We flailed around for a while and eventually someone found out that there is a known issue in Windows 7 Service Pack 1 that causes the Extensionless URL feature to interfere with the way ASP.NET parses URLs that would normally be handled by the Default Document setting.  This causes redirection to the Forms Authentication login page instead of the Default Document when the url is pointing to the root of the website.  The solution is to rewrite any requests made to the root of the website to a url that explicitly references the "defaultDocument".

Do this in the global.asx:

void Application_BeginRequest(object sender, EventArgs e)
{
    if (Request.AppRelativeCurrentExecutionFilePath == "~/")
    {
        // Get the defaultDocument filename from web.config.
        System.Xml.Linq.XDocument xDoc = 
            System.Xml.Linq.XDocument.Load(HttpContext.Current.Server.MapPath("~/Web.config"));

        string defaultDocumentName = 
            xDoc.Element("configuration")
            .Element("system.webServer")
            .Element("defaultDocument")
            .Element("files")
            .Element("add").Attribute("value").Value;

        // Rewrite the url.
        HttpContext.Current.RewritePath(defaultDocumentName);
    }
}

Hope this saves someone else some time.

Automated Testing

Automated testing as a strategy

Automated testing is a great and powerful tool for ensuring consistent code coverage, performing fast regression tests, validating builds and locating potential code problems.  The introduction of automated testing into your processes should give you a noted increase in productivity and QA throughput. While it is true that the automated testing will improve your coverage and productivity, it is not the answer to every 'QA bottleneck' issue in history.  Beware of jumping to conclusions about what automated testing will bring to the table and instead focus on where it gains the most value.  It will never replace human testing in its entirety.  It simply will not replace manual testing in aspects such as product exploration and environmentally varied user story testing.

Automated tests will remove much of the burden from a human resource by testing a single action or logical group of actions repeatedly.  That is the great gift of automated testing, but it cannot entirely take the place of a human with platform knowledge exploring the product for cause and effect testing. It is important to focus your testing investments where they will have the most return.

Mike Cohn created a test automation pyramid indicating a good break down of test investments.  I have taken that and applied it to the platforms we are dealing with daily.  This pyramid illustrates a healthy distribution of test investment for complete platform coverage and explains how the investment in tests should focus at the unit level and then reduce up through the application layers.

Unit Tests

Unit tests should be your highest investment in code coverage.  These tests are created by the developers as they write the corresponding code. In a SOA world, every line of code in a service should have a corresponding unit test.  This includes all methods, extensions, data contract validations, CRUD operations and authentication routines.  These should have a test created as part of the development process.  Even when a modification is made to existing service, the modification must have corresponding tests to prove it is functioning as requested.  By its nature, a unit test will inherently give your platform total code coverage at a very base level.  It is important that all unit tests deal with every possible code path within the method being tested.  Cyclomatic complexity limitation is a standard specific metric that should be enforced with scripts, tools or code reviews, but will keep your unit tests, as well as your methods, clear, concise and focused.  For the sake of focus, I am avoiding the CC soapbox for today. This flavor of automated testing gives you the most absolute code coverage and is the base for automated testing.

Integration Tests

Integration testing is created by QA automation employees and is meant to validate functionality of entire user stories.  This is where other considerations such as performance should be measured. Integration tests should be SOA tests, meaning that the integration test plays the role of the presentation layer in executing a chain of tasks comprising user stories. These tests are constructed to validate and measure larger functional items that consist of many small service calls. A good example of an integration test is a long story consisting of customer creation, order creation, payment creation, editing of account credits and debits and comparing the end result against known values for outcome, performance and A/R.  The integration test provides regression testing at the user story level and needs to be done to all USL and SDK-API methods, as they are the outer most points in the SOA.  This type of automated testing offers the most user story coverage and should comprise the primary basis for validating the quality and functionality of your platform.

UI Tests

The UI test is the process of ensuring a UI meets specifications. UI tests are created by QA automation employees and are simply a robotic replacement for a human moving through the platform and executing the elements within a UI.  UI testing is a valid and valuable platform for product stability, but is ultimately a less indicative test of total platform quality than the areas before it.

Human Testing

Human testing should be approached as primarily an exploratory testing methodology.  In this environment, a person familiar with the platform spends the time varying individual environment variables and testing the functionality of the platform.  This is the layer at which testing no longer reflects a series of repeatable actions and begins to reflect an individual creatively trying to find flaws in a system. This is a valuable test concerning ensuring that a platform will be stable when not being used correctly or encountering anomalies in the standard environment. 
If approached with the proper expectations and focus, automated testing can be the answer to many aspects of QA that seem slow and redundant.  If we ensure the proper level of investment at each area of testing, the platform coverage can be complete and result in a much faster and more thorough acceptance process.

Federation Architecture

While creating web applications with large, multi-tenant data sets, it becomes an immediate architectural need to plan for scaling data storage.  This plan cannot be an afterthought, but must be a paramount consideration at the outset of the project.  Performance and scalability issues on the data layer can create an array of issues that each manifests themselves as a poor user experience, and ultimately damage the brand of the application.  One such approach for handling this need is to scale horizontally using a technique known as 'Sharding'.  Sharding allows one to separate the rows of storage across multiple physical databases, which enables much scalability and should result in better performance on the data layer.  This process enables one to plan for scale, build for speed and control capacity.  Sharding also allows the operational aspect of the web application to scale, increase performance, and add additional capacity dynamically to the data layer with no downtime, which is vital to a thriving user base.

Federations are individual data partitions, which have their individual scheme centrally managed by a single distribution (or federation) scheme.  That scheme defines and controls the single keying mechanism for cross-partition distribution.  While a handful of data types are acceptable for the distribution key, I like the simplicity of a bigint as defining key (of any kind actually).

The individual partitions are members of the federation, each with their own schema.  As such, they are responsible for any inclusive subset of the values in a federated table covered by the data type of the federation distribution key.  The individual can be responsible for all of the values or a range of the values giving the architecture the ability to scale dynamically to match the current need.   While each partition has its own schema, the table keys correspond to the federation scheme. A federation member may also contain tables that are not part of the federation, known as reference tables.  Reference tables can be including in results that along with federation aware data.  It is important to note that each partition controls its own schema.  As such, it may or may not match the schema of other member partitions.

When building a federation plan, a paramount decision to make is deciding value upon which value to federate. I think the best practice may be to use a value that is meaningful to the data separation you are trying to achieve. In my world, the thing that makes the most sense is the customer or tenant identifier.  This gives us the ability to centrally reference all data for querying, yet provide each customer with what amounts to a singularly responsible and sovereign data set.

While sharding is a great solution for these types of application, it is important to understand the complexity that accompanies the sharding process. Depending on the individual implementation flavor, sharding may developers handle rollbacks, constraints, and referential integrity across tables when historically those items have been handled by the database itself.   It also makes joins, global searches and other high-level insight more difficult.  Even knowing the trade-offs being made for the ability to scale data, it is hard to argue a properly executed sharding strategy for serving multi-tenant data in a web-mobile application world.  The process checks all of the boxes required by the various user stories and operational concerns.

Sharding is a good example of a core belief of mine; It really should not matter how difficult or easy, how fancy or how simple a given technique or design is. The right answer should be the right answer.  You should not over-design because one thing seems too simple, nor should you under-design because it seems too hard.  The entirety of the platform truth should become self-evident and then pursued as the goal.

Write an image to the reponse of an aspx page.

I recently had the requirement to read image data from a memory stream and write it to the response of an aspx page.  This page was to be used as the source for an image control and should take query string values to get the image data (as part of a containing data contract) by an id and then write the image directly to the response.  Here is the code I ended up with:

public partial class DisplayImageThumbnail : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {

        String imageId= Request.QueryString["imageId"];
        imageId = imageId?? "-1";
        Bitmap image;       
        try
        {
           // get data contract containing image from service
           CustomImage csImg = ImageInteraction.GetImageById(imageId);
           image = csImg.ImageData;
        }
        catch
        {
           // show a 'no image' image in case of error or n/a
           image = (Bitmap)Bitmap.FromFile(Server.MapPath(String.Format("../Images/no_image.jpg"));
        } 
       
        // will render blank if no image.
        Response.ContentType = "image/jpeg";
        image.Save(Response.OutputStream, ImageFormat.Jpeg);
    }
}

Simple little method once I figured it out, but it seemed like it might help someone else in a similar situation.