OData , Atom and AtomPub

The Open Data Protocol (OData) is a protocol which standardizes the exposure and consumption of data. In times where data is being exposed at high rates and where consumers connect to more and more data endpoints, it’s important for clients to access these endpoints in a common way. OData builds on standards like HTTP, Atom, and JSON to provide REST access to controller based endpoints. Data is exposed as entities where each entity can be treated as an Http resource which makes it subject to CRUD (create, read, update, delete) and Patch operations.

Atom is way to expose feeds much the same way RSS does. Atom by itself allows only feed exposure. If you want to publish data, AtomPub (Atom publishing) provides this ability. AtomPub uses HTTP verbs GET, POST, PUT, and DELETE to enable data publishing.

This is not an implementation to be used in every situation obviously.  But it is an interesting flavor of having feed available data with real interaction.

ASP.NET Web API

The ASP.NET Web API framework was introduced recently into ASP.NET.  I touched on it some previously at an overview level, but would like to dig in a little more at this time to show the power of having a true Web API in your product.  

Overview

From a high level, leveraging the Web API framework in ASP.NET allows you to easily build HTTP based services geared at reaching browsers, desktop or mobile clients.  By creating a responsive layer of RESTful APIs for a product, the platform immediately becomes a much more adaptable and scalable solution.  The simple act of ensuring all 'real' application logic is handled behind a RESTful API call, an inherent separation of interests is manifested, which enables a more loosely couple application at every layer.

ASP.NET Web API Infrastructure

At its base, the ASP.NET Web API mechanism is a simple adaptation of an HTTP service.  Web API can be used along with WCF, but realistically, they are two means to create virtually the same thing.  The main aspect of a Web API layer is the ability to create an API project within a given solution that allows user stories to be executed via RESTful calls to the services.  Microsoft has a great tutorial for creating a simple Web API project as an introduction to the platform.  For those familiar with MVC, this exercise should be very straightforward and easily worked through.  In fact, as you work through this, it will probably become clear that there is very little difference here than a standard MVC application.  The main difference is that Web API uses the HTTP method, not the URI path, to select the action. But, you can also use MVC-style routing in Web API if that is more your style.

Dynamism

Leveraging an API infrastructure as a service layer for you web application gives you the ability to manifest any client on any platform without recreating application intelligence. Some of the main features of Web API allow for the creation of a dynamic service layer that can interact with multiple versions of the same platform, 3rd party vendors, customer integrations and most importantly, your own application.  A strong API will have an entry point to accommodate the same list of user stories that the main presentation layer in your application presents to a user.  The API layer should handle modeling, security, extensibility and scalability.  This allows for the consistent use of application logic and business rules across multiple consuming platforms.  API/SDK layers should be manifested by version, with all historic versions co-existing.  A single set of business logic and data access logic should persist across versions of API.  This allows forward movement without backward degradation.  By adhering to the rule of the eternal public interface, this is an achievable goal.

The 'Real' API

While the main focus of this post is the ASP.NET Web API offering, I would like to make a point regarding what an API should be in my opinion.  An API should be a complete manifestation of every user story that exists in your application.  The API should be a service layer that is not the data access layer, nor is it the main business logic layer.  It is the manifestation of User Stories.  This type of layer has been called the User Story Layer and the services contained within have been called Complex Services.  This is a logical regrouping, or coupling of specific business logic tasks to create a user story.  A user story should be completely executed within its single methodological representation, including input validation, security and access validation, activity logging and return values. All manifested user story methods should assume no knowledge of requirements, modalities, system or data flow from any client executing the method. Data access and business logic should never be directly exposed to an end user, third party or otherwise.  Data access and business logic should be separated from each other as well as from the API/SDK/USL.  This enables the ability to quickly couple existing sets of logic into a new user story without having to accommodate situational elements inside the business logic layer.  For more information on building a strong SOA, see some of my previous posts here.  For a theoretical discussion of the purpose behind SOA, check this post.  A framework that has been built with this decoupled approach of singularly responsible object residing behind globally accessible services will much more easily allow for productivity items such as Continual Integration, Continual Deployment, and fully automated testing.

Your First API Client

The short answer here is that it’s your application.  A robust API should first and foremost serve all logic to your own applications.  You have to create your API and then consume it.  This ensures that all changes made are not only accounted for in the API, but are placed there with intent.  You have to eat your own dog food.  In a web development and agile deployment environment, the ability to decouple code and responsibility of objects is the best friend you will ever have.  Approach your web design with intent and long term vision for things like scalability and performance.  Understand that you need to support multiple versions of your service layer and extend your data objects over multiple versions.  A properly architected system will ensure you are able to respond to changes in your market and your market requirements much more quickly than would be the case otherwise.

Federation Architecture

While creating web applications with large, multi-tenant data sets, it becomes an immediate architectural need to plan for scaling data storage.  This plan cannot be an afterthought, but must be a paramount consideration at the outset of the project.  Performance and scalability issues on the data layer can create an array of issues that each manifests themselves as a poor user experience, and ultimately damage the brand of the application.  One such approach for handling this need is to scale horizontally using a technique known as 'Sharding'.  Sharding allows one to separate the rows of storage across multiple physical databases, which enables much scalability and should result in better performance on the data layer.  This process enables one to plan for scale, build for speed and control capacity.  Sharding also allows the operational aspect of the web application to scale, increase performance, and add additional capacity dynamically to the data layer with no downtime, which is vital to a thriving user base.

Federations are individual data partitions, which have their individual scheme centrally managed by a single distribution (or federation) scheme.  That scheme defines and controls the single keying mechanism for cross-partition distribution.  While a handful of data types are acceptable for the distribution key, I like the simplicity of a bigint as defining key (of any kind actually).

The individual partitions are members of the federation, each with their own schema.  As such, they are responsible for any inclusive subset of the values in a federated table covered by the data type of the federation distribution key.  The individual can be responsible for all of the values or a range of the values giving the architecture the ability to scale dynamically to match the current need.   While each partition has its own schema, the table keys correspond to the federation scheme. A federation member may also contain tables that are not part of the federation, known as reference tables.  Reference tables can be including in results that along with federation aware data.  It is important to note that each partition controls its own schema.  As such, it may or may not match the schema of other member partitions.

When building a federation plan, a paramount decision to make is deciding value upon which value to federate. I think the best practice may be to use a value that is meaningful to the data separation you are trying to achieve. In my world, the thing that makes the most sense is the customer or tenant identifier.  This gives us the ability to centrally reference all data for querying, yet provide each customer with what amounts to a singularly responsible and sovereign data set.

While sharding is a great solution for these types of application, it is important to understand the complexity that accompanies the sharding process. Depending on the individual implementation flavor, sharding may developers handle rollbacks, constraints, and referential integrity across tables when historically those items have been handled by the database itself.   It also makes joins, global searches and other high-level insight more difficult.  Even knowing the trade-offs being made for the ability to scale data, it is hard to argue a properly executed sharding strategy for serving multi-tenant data in a web-mobile application world.  The process checks all of the boxes required by the various user stories and operational concerns.

Sharding is a good example of a core belief of mine; It really should not matter how difficult or easy, how fancy or how simple a given technique or design is. The right answer should be the right answer.  You should not over-design because one thing seems too simple, nor should you under-design because it seems too hard.  The entirety of the platform truth should become self-evident and then pursued as the goal.

Authorization Using WIF 4.5

I'd like to continue the topic of claims-based security using WIF 4.5, and talk about the authorization model and its options at a deeper level. First, just to ensure we are all using the same terminology when discussing items, I want to clarify "authorization" when I use it in this context. Authorization and authentication are still widely confused when discussing overall security methodology. They are both aspects of security, but they play completely different roles in the process.

Authorization vs. Authentication

Authentication is really just the process of having users log in so you can ensure they have the right to access the application. Authentication is the concept at work when you log in to any web application at all. By having an approved and active user account, and entering the proper credentials, the user is authenticated to use the application. In most cases, users are authenticated prior to authorization.

Authorization is the mechanism by which we evaluate whether an authenticated user has access to a specified modality (url, service, module or data element) within the application to which they are authenticated. For instance, an authenticated user may be an administrator which makes them authorized to change settings, post content, etc. While another authenticated user may be a guest, which makes them authorized to read public content. In its most simple form, authentication is who you are and authorization is what you are allowed to do.

Claims-based Authorization

As we discussed before, claims-based authorization is the approach where you write code to allow or disallow access based on logic that checks data called 'claims'. Remember that in the case of a role-based authorization, the only claim we actually used was the claim a user made about his or her roles. A role claim was used for us to decide if the current user 'is a' specific role. Let's walk through making access decisions using a claims-based authorization approach.

1. A user arrives at your application and needs to be authenticated.
2. WIF forwards the user to your identity provider.
3. Once the user is authenticated, the original request is made again, but now there is a new security token attached to the request which contains data representing the user by using claims representing each granular piece of data. WIF attaches these new claims to the principal that represents the user so it can be referenced quickly and easily.
4. Your application checks the claims via code to ensure the claims are met. These checks can be made via code, service calls, database, home-grown rules, or the native ClaimsAuthorizationManager.
5. Your application decides whether or not to allow the request based on the claims:logic check.
6. Your application grants the request if the outcome of your check is true and denies it if false.

The ClaimsAuthorizationManager is a new tool in WIF 4.5 that seems to be useful for black-boxing the logic for any claims-based authorization in your applications. It allows you to sniff incoming requests and wrap access with a check to custom logic which will then make authorization decisions based on the incoming claims. Much like any other good design principle, abstracting your authorization logic can only add to your applications ability to be dynamically responsive to changing authorization requirements. If you have to customize or change your authorization rules, using ClaimsAuthorizationManager will not affect the core application code base. Anytime you can separate logic into a stand-alone process and create a producer-consumer relationship, I think you are creating dynamism in the application. This is just another means for injecting flexibility to potentially deal with unforeseen changes without degrading the 'meat and potatoes' of your application. The ClaimsAuthorizationManager appears to be a great opportunity to allow authorization to be black-boxed inside a singularly purposed service entity, which can position your application to leverage centralized logic in this area.

Role-based Authorization

Role-based authorization is implemented by assigning users to user roles that have been defined for the application. This is the example that I used when talking about an administrator or a guest. Both of those are considered roles. When using this methodology, users are assigned 1:n roles within the system which can be checked at run time for derivation of access rights. You can check the roles of the user in a few different ways to ensure they are authorized for access. 

• Checking IPrincipal.IsInRole(“RoleToCheck”). This is probably the most straight-forward and widely-used method. Since it returns a boolean indicating membership in the role, you can use it in your conditional statements at any place in your code as a security check.
• Using PrincipalPermission.Demand(). When you make the Demand() call, the application will throw an exception if the demand is not met by the roles. This flows into standard try-catch-finally blocks and makes authorization just another exception in the block. Be warned that exceptions are much more expensive from a performance standpoint when compared to the IsInRole() method.
• Using the <authorization> section in web.config. This really only works if you are blocking entire URLs against roles. This is the most strict, blanket level authentication so far. The upside is that it requires no code changes for implementation since it is part of the configuration file.
• Using [PrincipalPermission(SecurityAction.Demand, Role = “RoleToCheck”)] attributes. The declarative method cannot be used in code blocks or within the service/method implementations as it is an actual attribute of the method itself. The method will throw an exception if the authenticated user doesn't belong to the role being demanded.

As mentioned above, when you call the IsInRole() method, the system checks behind the scenes to see if the current user has that role. In claims-aware applications, the role is expressed by a role claim type that should be available in the token. When a user is authenticated, the role claim can be issued by the identity provider STS or by a federation provider such as the Windows Azure Access Control Service (ACS). You can also turn arbitrary claims into role-type claims using the ClaimsAuthenticationManager component of WIF. This allows requests to be intercepted when an application launches, allowing you to inspect the tokens and even transform them by adding, changing or removing claims.

Claims-Based Identity and Authorization

As a follow-up to the previous post about security, I am examining a claims-based security model within WCF using the Windows Identity Foundation (WIF). In a claims-based application, a user is represented by a set of claims. The basic idea is that an external service is configured to give you any relevant information about the user as part of each request. This also includes some assurance that the identity data you receive comes from a trusted source by using some flavor of cryptography. This really flows into a good SOA design because you are basically decoupling your application from the logic of authentication, the storage and protection of user names, passwords and emails, and forced integration with in-house identity systems. In a claims-based paradigm, your application is making decisions for access and security using information from the authenticating system.

Claims

Claims are really just pieces of data associated with the currently authenticated user. These 'claims' from the authenticating service can consist of items such as the user name, email, security group membership, etc. When your application receives these claims, it enables you to cater the availability of content to the authenticated user. A very important point on which to be clear is that the claims made from the issuing service are only as trustworthy as that service itself. For example, you trust a claim made by your company’s domain controller more than you trust a claim made by, for example, Facebook. In that regard, WIF represents claims with a Claim type, which has an Issuer property that allows you to find out who issued the claim.

Tokens

The user delivers a set of claims to your application along with a request. In a web service, these claims are carried in the security header of the SOAP envelope. In a browser-based web application, the claims arrive through the HTTP request from the browser. The claims, regardless of the arrival method, are serialized, which is where why we need security tokens. A security token is really just a serialized set of claims that is digitally signed by the issuing authority. The signature is needed because it guarantees the issuing authority has generated the attached claims, which validates their authenticity. In other scenarios where that kind of security isn’t needed, you can use unsigned tokens. One of the core features in WIF is the ability to create and read security tokens. WIF and the .NET Framework handle all of the cryptographic work, and present your application with a set of claims that you can read.

Relying Party

When you build an application that relies on claims, you are building a relying party (RP) application. Synonyms for an RP include “claims-aware application” and “claims-based application.” Both web applications and web services can be RPs. An RP is nothing more than a consumer of tokens issued by an STS who reads the claims from tokens for use in security or access as it relates to the identity of the user. WIF offers inherent functionality to help you build RP applications.

Issuing Authority

It seems there are endless types of issuing authorities, but we really want to focus on security tokens that contain claims. This issuing authority is another application or service that is tasked with the issuance security tokens. This logic is responsible for having the insight needed to be able to issue the proper claims given the specific application and the user making the request, and might ultimately just be a pass-through to other services to receive claims and authenticate users.

Security Token Service (STS)

A security token service (STS) is the service that creates, signs, and issues any security tokens based on the WS-Trust and WS-Federation protocols. These protocols are really difficult to implement on a stand-alone basis, but WIF seems to handle the vast majority of this work. It appears that it is much easier to get STS up and running by leveraging WIF. You can use a pre-built STS such as Active Directory® Federation Services (AD FS) 2.0, a cloud STS such as a Windows Azure Access Control Service (ACS), or, if you want to issue custom tokens or provide custom authentication or authorization, you can build your own custom STS using WIF. The following is an example from Microsoft of a claims-based system.

This diagram shows a Web site (the relying party application, RP) that has been configured to use WIF for authentication and a client, a web browser, that wants to use that site.

1. When an unauthenticated user requests a page their browser is redirected to the identity provider (IP) pages.
2. The IP requires the user to present their credentials, e.g. username/password.
3. The IP issues a token back to that is returned to the browser.
4. The browser is now redirected back to the originally requested page where WIF determines if the token satisfies the requirements to access the page. If so a cookie is issued to establish a session so the authentication only needs to occur once, and control is passed to the application.

It seems that leveraging WIF as an STS provider is an easily achievable way to use a secure and standardized methodology of application security.

SOA and Security

We have talked about WCF at great length, but have focused mainly on the scalability and dependability of the Service-Oriented Architecture (SOA) itself.  One of the topics we haven't gotten into at a sufficient depth is security and service access management as an integral part of the SOA design.  I'd like to explore some of the options available for securing your services while adhering to the good design principles of SOA.  But first, I would like to discuss what it means to have security built in to your SOA.  We know that a well-designed SOA should be an architecture of loosely coupled components that can be distributed across platform, technology, and physical environments. Service components can be abstracted, grouped and presented to provide a single process, or to provide more complex user stories for a consuming application. Services are the preferred communication technique across application boundaries, including platform, deployment, and trust boundaries.  The key attributes of SOA are:

• Interoperable. Components can be interoperable across platform and technology boundaries.  In fact, a well designed service should be totally agnostic to the technologies interacting with it.
• Autonomous. Services are exposed as autonomous components that can be versioned and managed independently.  As with all good platform design, think of each service as a black box.
• Abstracted. Services should be abstracted at a granular enough level so they can be grouped by another service or application to perform more complex operations as a single method.  This is what I often call a user story service layer.  Business logic can be abstracted and presented as specific component-like stories for targeted consumption.
• Interface based. Interfaces are defined by message contracts and schemas. Operation calls and parameters are passed in XML message envelopes.  Stick to agreed upon standards here for maximum scalability and interoperability.
• Location agnostic. Service components can be consumed from the same machine or distributed to remote machines. The service interface and logic is independent of the transport and protocol used to access the service.  This gives you many points to scale and inject redundancy for performance and dependability.
• Discoverable. Services publish their interface in such a way that client applications can discover them and generate a proxy mechanism for consumption.

I try to adhere to the agreed upon four tenets of SOA from Microsoft architect Don Box:

• Boundaries are explicit. Services are black-boxed and will never allow direct internal visibility.  All interaction comes from agreed upon messaging and interaction methods.
• Services are autonomous. Each service is viewed autonomously with a complete disregard for any client activities or accompanying services.  While interaction is allowed and obviously required, see the first tenet.  These services should not be intimately aware of each other.
• Services share schemas and contracts, not class. Services share contracts and schemas to communicate.  This is analogous to a menu in a restaurant.  All of the inner workings are abstracted to a set of choices and communicated with an agreed upon contract. 
• Compatibility is based upon policy. Policy in this case means definition of transport, protocol, security, etc.  This is primarily where we are focusing today.

To focus in on the compatibility tenet, let's delve into security at a deeper level. Security is fundamentally about protecting assets. Assets may be physical items such as code sets, methods or a database, but also can be more nebulous, such as your brand.  It is important to recognize that security is a commitment, not a software feature, as it is a never-ending process. As you design services and applications, you need to identify potential threats and understand that each threat presents a degree of risk. Security is basically about mitigating those risks via restrictions and/or counter measures. When done effectively,  security is a combination of people, process, and technology.

Security is comprised primarily of the following ideals:

• Authentication. Authentication is, at its base, the ability for a system to know who the user is.  It is the process of uniquely identifying the users of applications and services. Keep in mind that users of applications and services may be people, other services, processes, or computers. When reading security documentation or writing code, these 'users' are called Principles.
• Authorization. Authorization is the equivalent of a key card.  Just as a key card gives its holder permission to enter certain rooms in a building, authorization manages the resources and operations that the authenticated client is permitted to access. These resources can include operations, physical storage (as granular as tables and rows), registry keys and other  configuration data. Operations are basically equivalent to the methods at any given layer of the SOA.
• Auditing. Effective auditing and logging is the key to non-repudiation. Non-repudiation at its base is evidence.  If you have implementing effective auditing, a user or process will be unable to deny its activities. For example, in a medical system, auditing is required to make sure that a user cannot deny changing information in a specific patient chart.
• Confidentiality. Confidentiality, or privacy, is the process of making sure that all data is private and confidential and cannot be viewed by unauthorized users who attempt to monitor the traffic across a network. Encryption is the most used method of confidentiality.  Primarily, you must ensure that the only users and processes who can view specific data, are the ones for whom it is intended.
• Integrity. Integrity is the guarantee that data is protected from modification, be it accidental or malicious. Like privacy, integrity is a huge deal, particularly when there is sensitive data being passed across networks. Integrity for data in transit is typically provided by using hashing and message authentication codes.
• Availability. From a security perspective, availability means that systems remain available for legitimate users.  There are some malicious individuals who will attempt to bring down your services and thus cause your real users to be unable to leverage the system.  The denial of service attack (DoS) is aimed at simply over-running your system and bringing it down. 

Some keys to building secure services include the following:

• Identify the targets for security. You should think through all of the objectives you have for security. User stories are a good tool for gaining insight into the natural roles and levels of security that come from usage of your system.  Identify all assets that need security as well as the granularity to which security is applied.
• Understand the possible threats. Try to identify the threats which are relevant for your scenarios and context. This is called 'Threat Modeling' and is a good practice which helps you identify the realistic  threats to your system. Your security targets will help you prioritize your threats and vulnerabilities. Using the threat model, architects and developers should address vulnerabilities, and QA should ensure that the developers have addressed the vulnerability.
• Stay away from 'roll your own' security. Proven principles, patterns, and practices are the best place to start building a security model.  By using time-tested, proven principles, patterns, and practices, you can eliminate entire groups of security problems without needing to discover the specific vulnerability in your identification process. You should take advantage of the experience of others when it comes to security.  That being said, proven principles, patterns, and practices are a great starting point.  But like all other architectural paradigms, security is fact dependent and you should not just implement something off-the-shelf without going through the identification and threat processes.
• Commit to security as an eternal process.  Security should be of paramount consideration at every stage of a service, application or software development life cycle. At every stage, targets should be defined, threats should be modeled and mitigating actions should be taken.  Security is not and will never be a 'set it and forget it' item.

I am going to dig deeper into some of the possible mechanisms for securing WCF services in a web environment in the near future.  As part of a current project, I am spending a lot of mental energy looking into the options.  I will post some of the opportunities and how to leverage them in the coming days and weeks.

.NET 4.5 - Simplified WCF Configuration

A WCF configuration file is basically a markup file used to communicate to the hosting environment how a WCF service is configured and should behave.  The configuration files have historically required an exhaustive explanation of each aspect of configuration and behavior.  To illustrate this, consider a standard config file.  Notice that the file contains a <system.serviceModel> section which contains a <service> element for each service hosted.  The <service> element contains a listing of <endpoint> elements that outline the endpoints for each service and a set of service behaviors.  The <endpoint> elements specify the address, binding and contract exposed by the endpoint, and optional binding and endpoint configuration.  The <system.serviceModel> section also contains a <behaviors> element that allows you to specify service or endpoint behaviors.  The following example shows the <system.serviceModel> section of a configuration file.

<system.serviceModel>
  <behaviors>
    <serviceBehaviors>
      <behavior name="MyServiceBehavior">
        <serviceMetadata httpGetEnabled="true">
        <serviceDebug includeExceptionDetailInFaults="false">
      </behavior>
    </serviceBehaviors>
  </behaviors>
  <bindings>
   <basicHttpBinding>
      <binding name=MyBindingConfig"
               maxBufferSize="100"
               maxReceiveBufferSize="100" />
   </basicHttpBinding>
   </bindings>   <services>
    <service behaviorConfiguration="MyServiceBehavior"
             name="MyService">
      <endpoint address=""
                binding="basicHttpBinding"
                contract="ICalculator"
                bindingConfiguration="MyBindingConfig" />
      <endpoint address="mex"
                binding="mexHttpBinding"
                contract="IMetadataExchange"/>
    </service>
  </services>
</system.serviceModel>

In 4.5, the configuration has been simplified by no longer requiring a <service> element.  When the <service> section is absent, or a <service> section defines no endpoints (and your service does not programmatically define any endpoints), default endpoints with default configurations are automatically added to each service.  These defaults are added for each service base address and for each contract implemented by the service.  In the default endpoints, the endpoint address relates the base address, the binding is determined by the base address scheme and the contract is the one implemented by the service.

To take this further, if you have no need to specify endpoints, modify behaviors or customize the binding, The config file is not needed at all.  For example, if a service implements x contracts and the host enables both HTTP and TCP, the service host creates x*2 default endpoints as a straight contract:endpoint.  To create default endpoints the service host must know what bindings to use.  These settings are specified in a <protocolMappings> section within the <system.serviceModel> section.  The <protocolMappings> section contains a list of transport protocol schemes mapped to binding types.  The service host uses the base addresses passed to it to determine which binding to use.  The following example uses the <protocolMappings> element.

<protocolMapping>
  <add scheme="http"     binding="basicHttpBinding" bindingConfiguration="MyBindingConfiguration"/>
  <add scheme="net.tcp"  binding="netTcpBinding"/>
  <add scheme="net.pipe" binding="netNamedPipeBinding"/>
  <add scheme="net.msmq" binding="netMSMQBinding"/>
</protocolMapping>

Service behaviors are configured for the default endpoints by using anonymous <behavior> sections within <serviceBehaviors> sections.  Any unnamed <behavior> elements within <serviceBehaviors> are used to configure service behaviors.  For example, the following configuration file enables service metadata publishing for all services within the host.

<system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior>
          <serviceMetadata httpGetEnabled="True"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>    
 </system.serviceModel>

Here is the configuration file from the top of this post modified to use the simplified model.

<system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior>
          <serviceMetadata httpGetEnabled="True"/>
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
   <bindings>
     <basicHttpBinding>
          <binding maxBufferSize="100"
                   maxReceiveBufferSize="100" />
     </basicHttpBinding>
   </bindings>   
   <protocolMapping>
      <add scheme="http" binding="basicHttpBinding" />
   </protocolMapping>
  </system.serviceModel>

Multi-tenant WCF Services

Hosting a multi-tenant application using WCF
While doing some long-term architecture design, the task was given to work through a cloud-based application.  Our business requirements have resulted in the need for a multi-tenant application built on WCF services that can share physical instances for multiple tenants. The first thing that jumps out in this framework for me is that these services must not contain state level variables that would directly relate to any single tenant. In this model, for instance, a BLL type service should not contain a static list of 'droplist' items to be consumed by the software unless the listing was tenant/customer agnostic. This breakdown gives us the ability to scale at any point in the SOA, based on load or any other need that may present itself. The gain is that you are not locked into a 1:1 tenant:service instance relationship which allows for more host cost effective scaling. In our model, based on the type of data stored, we have decided to go with a separate catalog for each tenant. This point works for our model and while I know there are other models that work effectively, this is really not the target of the current post.

Building the flow and responsibilities 
The first obstacle presented was how to point a tenant at the corresponding catalog without having the SOA be inherently aware of the tenant. For our model, we have the user authenticating against a centralized database and service purposed exclusively for housing tenant information. This enables us to assume that a client will be self-aware and could be provided with their tenant information to some degree prior to engaging the full SOA of the application. The idea being that if we can have the service dynamically recognize the tenant based on the tenant self-identifying at the time of messaging, we can achieve a loosely coupled environment for services to perform with a disregard for the actual tenant entity.

Communicating tenant information
While thinking through a means to provide tenant information to the services, there are a couple of possibilities that come to mind. One could simply provide all of the required information as parameters to each service call.  Would it work? Yes. Is it a good idea? No. There are many reasons this is not a good solution. One concern is the added difficulty each client application would encounter trying to consume the SOA. Plus, the SOA would be confusing, difficult to implement and not very readable as a programmer. To me, building your SOA must always consider the consuming applications and the ease with which the services can be consumed as a high priority. Those alone point us directly to the messaging interface for the service. My initial thought was to create custom incoming and outgoing message header properties to allow C# windows clients to attach the properties directly via their client proxy and then dynamically read them out in the service layer. This allows all additional difficulty to be encapsulated within the client proxy/channel factory classes. This also allows non-windows users to dynamically build message headers and attach them to the calls coming into WCF while allowing the service to leverage a single implementation to pull the information out of the message. I needed to create a proof of concept application to illustrate this. I was wrapped up in another project at the time, so ToddM took the project on and delivered an illustrative project. (Thanks, Todd). An overview of the successful prototype is as follows. First, you have to attach the outgoing behavior message header at the channel factory level in C#. This requires the creation of CustomServiceOutgoingBehvaior and CustomServiceIncomingBehvaior classes based on IEndpointBehavior and Attribute, IServiceBehavior respectively. These are then attached to the DataChannel and inspected on either side of the communication. In an effort to make this post shorter and more readable, I have removed code that doesn't directly support the main idea. To be clear, these snippets on their own are not enough.

public class DatabaseInfo
{
  public String Catalog { get; set; }
  public String Instance{ get; set; }
}
public static class Tenant
{
    public static String TenantID { get; set; }
    public static String AuthenticatedUser { get; set; }
}

public class CustomClientOutgoingBehvaior : IEndpointBehavior
{
   void IEndpointBehavior.ApplyClientBehavior(ServiceEndpoint endpoint,
              ClientRuntime clientRuntime)
    {
        clientRuntime.MessageInspectors.Add(new CustomClientOutgoingMessageInspector());
    }

}

public class CustomServiceOutgoingBehvaior : IEndpointBehavior
{
    DatabaseInfo info = null;

    public CustomServiceOutgoingBehvaior(DatabaseInfo info)
    {
        this.info = info;
    }

  
    void IEndpointBehavior.ApplyClientBehavior(ServiceEndpoint 
              endpoint, ClientRuntime clientRuntime)
    {
        clientRuntime.MessageInspectors.Add(new CustomServiceOutgoingMessageInspector(info));
    }
}

public class CustomServiceIncomingBehavior : Attribute, IServiceBehavior
{
    void IServiceBehavior.ApplyDispatchBehavior(ServiceDescription
           serviceDescription, ServiceHostBase serviceHostBase)
    {
        foreach (ChannelDispatcher channelDispatcher in 
         serviceHostBase.ChannelDispatchers)
        {
            foreach (var endpointDispatcher in channelDispatcher.Endpoints)
            {
                endpointDispatcher.DispatchRuntime.
                     MessageInspectors.Add(new 
                     CustomServiceIncomingMessageInspector());
            }
        }
    }
}
public class CustomClientOutgoingMessageInspector : IClientMessageInspector
{
   object IClientMessageInspector.BeforeSendRequest(ref Message 
     request, IClientChannel channel)
    {
        var messageHeader = new MessageHeader<String>(Tenant.TenantID);
        var untypedMessageHeader = 
               messageHeader.GetUntypedHeader("TenantID", "Namespace.Shared");
        request.Headers.Add(untypedMessageHeader);

        messageHeader = new MessageHeader<String>(Tenant.AuthenticatedUser);
        untypedMessageHeader = 
              messageHeader.GetUntypedHeader("AuthenticatedUser", "Namespace.Shared");
        request.Headers.Add(untypedMessageHeader);

        return null;
    }
}

public class CustomServiceOutgoingMessageInspector : IClientMessageInspector
{
    public DatabaseInfo DatabaseInfo { get; set; }

    public CustomServiceOutgoingMessageInspector(DatabaseInfo info)
    {
        DatabaseInfo = info;
    }
  
    object IClientMessageInspector.BeforeSendRequest(ref Message request, IClientChannel channel)
    {
        if (DatabaseInfo == null)
            return null;

        var messageHeader = new MessageHeader<String>(DatabaseInfo.Catalog);
        var untypedMessageHeader = 
            messageHeader.GetUntypedHeader("Catalog", "Namespace.Shared");
        request.Headers.Add(untypedMessageHeader);

        messageHeader = new MessageHeader<String>(DatabaseInfo.Instance);
        untypedMessageHeader = 
            messageHeader.GetUntypedHeader("Instance", "Namespace.Shared");
        request.Headers.Add(untypedMessageHeader);

        return null;
    }
}

 

Then, with a simple function added to a base class for your WCF services, you will be able to pull this information back out as follows:

protected Tuple<DatabaseInfo, Tenent> GetInfoFromHeader()
{
    DatabaseInfo dbInfo = new DatabaseInfo();
    Tenant tenantInfo = new Tenant();

    Int32 i = OperationContext.Current.IncomingMessageHeaders.FindHeader("Catalog", "Namespace.Shared");

    if (i != -1)
        dbInfo.Catalog = OperationContext.Current.IncomingMessageHeaders.GetHeader<String>(i);

    i = OperationContext.Current.IncomingMessageHeaders.FindHeader("Instance", "Namespace.Shared");

    if (i != -1)
        dbInfo.Instance = OperationContext.Current.IncomingMessageHeaders.GetHeader<String>(i);

    i = OperationContext.Current.IncomingMessageHeaders.FindHeader("TenantID", "Namespace.Shared");

    if (i != -1)
        tenantInfo.TenantID = OperationContext.Current.IncomingMessageHeaders.GetHeade<String>(i);

    i = OperationContext.Current.IncomingMessageHeaders.FindHeader("AuthenticatedUser", "Namespace.Shared");

    if (i != -1)
        tenantInfo.AuthenticatedUser = OperationContext.Current.IncomingMessageHeaders.GetHeader<String>(i);


    return new Tuple<DatabaseInfo, Tenent>(dbInfo,tenantInfo);
}

 

At this point, you will be able to pull out information from the message header and perform many routines based on a tenant entity without adding state to the server. With this information, you can build connection strings for each DAL method pointing to the correct catalog and instance based on tenant.  You can also perform user security functions based on the AuthenticatedUser passed from the client.  As you can see, this is very flexible and would scale out to any values that needed to be communicated along with the messages to the service.

REST WCF Feed Service with Dynamic Response

Much like the project in which we built a WCF dynamic response service allowing for JSON, XML or DataContract serialization, this project is based on dynamic response capabilities.  In this project, we will continue with the Album services created previously and extend them to dynamic response as a syndicated feed.  This RESTful service will dynamically respond with either ATOM or RSS syndication based on the request type.

I will accomplish this using two different examples of response, one with a return type of System.ServiceModel.Channels.Message and another with a return type of SyndicationFeedFormatter.  First, we need to syndicate a feed based on our data repository.  I am including the ability to filter our feed based on genre and artist so we can access virtual sub-feeds with a uri.

private SyndicationFeed CreateAlbumFeed(String genre = "all", String artist = "all")

{

    SyndicationFeed feed = new SyndicationFeed

    {

        Title = SyndicationContent.CreatePlaintextContent("The album listing"),

        Description = SyndicationContent.CreatePlaintextContent("Dynamic feed response from single WCF"),

        LastUpdatedTime = DateTime.Now,

        Items = from a in DataRepository.Albums()

                where ((genre.ToLower().CompareTo("all") == 0)||(a.Genre == genre)) &&

                ((artist.ToLower().CompareTo("all") == 0) || (a.Artist == artist))

                select new SyndicationItem

                {

                    LastUpdatedTime = DateTime.Now,

                    Title = SyndicationContent.CreatePlaintextContent(a.Artist+": "+a.Name),

                    Content = SyndicationContent.CreateXmlContent(a)

                }

    };

    return feed;

}

This function will do nothing more than create a syndication feed from our album listing based on a couple of parameters.  Next, we need to add interface options for retrieving the feed.

 [OperationContract]

 [WebGet(UriTemplate = "/Feeds/Albums"), 

         Description("Returns an Atom or RSS feed of albums")]

 System.ServiceModel.Channels.Message GetAlbumFeed();

 [OperationContract]

 [WebGet(UriTemplate = "/Feeds/Genre/{genre}"), 

         Description("Returns an Atom or RSS feed of albums by genre")]

 SyndicationFeedFormatter GetGenreFeed(String genre);

 [OperationContract]

 [WebGet(UriTemplate = "/Feeds/Artist/{artist}"), 

         Description("Returns an Atom or RSS feed of albums by artist")]

 SyndicationFeedFormatter GetArtistFeed(String artist);

The return type of System.ServiceModel.Channels.Message is very generic and allows for you to simply build the response of a web request.  While generic and flexible, it does limit you by not allowing parameters on the REST uri.  In order to allow a SyndicationFeedFormatter return type, we must ensure that the service can serialize it as a result.  This is achieved by making it a known type for the service by decorating the interface as follows.

[ServiceKnownType(typeof(Atom10FeedFormatter))]

[ServiceKnownType(typeof(Rss20FeedFormatter))]

At this point, it comes down to dynamically formatting your response.  We do this by implicitly checking the Content-Type value of the request header in the body of our service methods.  Here is one example:

public SyndicationFeedFormatter GetGenreFeed(String genre)

{

    SyndicationFeed feed = CreateAlbumFeed(genre: genre);

    if (WebOperationContext.Current.IncomingRequest.Headers.Get("Content-Type") != null)

    {

        if (WebOperationContext.Current.IncomingRequest.Headers.Get("Content-Type").ToLower().Contains("application/atom"))

            return new Atom10FeedFormatter(feed);

        else

            return new Rss20FeedFormatter(feed);

    }

    return new Rss20FeedFormatter(feed);

}

This dictates how the feed will be formatted upon return from the function.  You can check the output using fiddler to see the message itself, as well as configure the content type for dynamism.

 Now we are able to browse to our URI and see a feed for all albums, albums by artist and albums by genre. 

The added ability to select the method in which the client consumes your feed is valuable to ensure all consumers are included.  Download the project here

Combo RESTful WCF with Windows Service Hosting and Dynamic Response Format

The Goal
I have been working through the plausibility of multipurpose WCF services that are hosted in a windows hosting environment.  As part of my research, I wanted to prove that you could host a single service as a standard service to be consumed by a client proxy and scale it out from there to webHTTP and RESTful behavior.  The biggest goal I had was to make the service dynamically reply in the same format in which it was consumed without having a drawn out implementation to handle the messaging.

The Project
After some reading and messing around I have completed the following project which does exactly what I had hoped.  This single WCF implementation will allow for a straight service call, a RESTful call and respond dynamically with xml or json to the REST query based on the content type value of the request header.  All while being hosted in a windows service that can be dynamically deployed.  I came up with a very simple project that allows for interaction with a listing of albums.  The example is simple, but the devil of this was not in the actual data elements, so I flagged the scalability of that as irrelevant to the goal of illustrating the mechanism.

The Services
I created a basic WCF service to perform CRUD on my album repository. You will notice the service decorations on the interface indicating both an operation contract and a WebInvoke/WebGet behavior and template.

[ServiceContract(Name = "AlbumContract", 

     Namespace = "RESTCombo.Services", 

     SessionMode = SessionMode.Allowed)]

public interface IAlbumSvc:IMetadataExchange

{

    [OperationContract]

    [WebGet(UriTemplate = "/Albums/{id}"), 

        Description("Returns the album with the passed ID")]

    Album GetAlbum(String id);

    [OperationContract]

    [WebGet(UriTemplate = "/Albums"), 

        Description("Returns the entire list of albums")]

    List<Album> GetAlbums();

    [OperationContract]

    [WebInvoke(Method = "POST", UriTemplate = "/Albums"), 

         Description("Adds a new album to list of albums")]

    void AddAlbum(Album album);

    [OperationContract]

    [WebInvoke(Method = "PUT", UriTemplate = "/Albums"), 

         Description("Updates an existing album")]

    void UpdateAlbum(Album album);

    [OperationContract]

    [WebInvoke(Method = "DELETE", UriTemplate = "/Albums/{id}"), 

         Description("Removes an album from list of albums")]

    void DeleteAlbum(String id);

}

The WebGet and WebInoke decoration allows the services to respond as a RESTful WCF service based on the URI template.  The service implementation should be decorated as follows:

[ServiceBehavior(Name = "RESTCombo.Services.AlbumSvc", 

    ConcurrencyMode = ConcurrencyMode.Single, 

    InstanceContextMode = InstanceContextMode.Single,

    IncludeExceptionDetailInFaults = true)]    

[AspNetCompatibilityRequirements(

    RequirementsMode=AspNetCompatibilityRequirementsMode.Allowed)]

public class AlbumSvc : IAlbumSvc

I'm going to skip posting the implementation of the services themselves as they are actually irrelevant to the discussion.  The entire project is attached at the end of the post if you would like to review the code.

The Configuration
While most of the configuration is fairly straight-forward, there are a couple of items worth pointing out.  Notice the multiple bindings for the single service.  Each must respond on its own port.  You can have as many bindings as needed up to one per protocol.  The webHttp endpoint behavior is vital to this mechanism.  helpEnabled allows users to use the '/help' switch at the end of a query to get a service overview page as shown here.

defaultOutgoingResponseFormat is our selection for the default response to a webHttp request. automaticFormatSelectionEnabled allows the response to dynamically detect the request format and respond in kind.

<?xml version="1.0"?>

<configuration>

  <startup>

    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>

  </startup>

  <system.serviceModel>

    <services>

      <service name="RESTCombo.Services.AlbumSvc">

        <host>

          <baseAddresses>

            <add baseAddress="http://localhost:8080/AlbumSvc/" />

            <add baseAddress="net.tcp://localhost:2122" />

          </baseAddresses>

        </host>

        <endpoint  binding="webHttpBinding" contract="RESTCombo.Services.IAlbumSvc"

                  bindingConfiguration="RESTBindingConfiguration" 

                   behaviorConfiguration="RESTEndpointBehavior"/>      

        <endpoint address="net.tcp://localhost:2122/AlbumSvc/" binding="netTcpBinding"

                  contract="RESTCombo.Services.IAlbumSvc"/>

      </service>

    </services>    

    <bindings>

     <webHttpBinding>

        <binding name="RESTBindingConfiguration">

          <security mode="None" />          

        </binding>

      </webHttpBinding>      

      <netTcpBinding>

        <binding name="DefaultBinding">

          <security mode="None"/>

        </binding>        

      </netTcpBinding>

    </bindings>

    <behaviors>      

      <endpointBehaviors>

        <behavior name="RESTEndpointBehavior">           

          <webHttp helpEnabled="true" defaultOutgoingResponseFormat="Xml"

                   automaticFormatSelectionEnabled="true"/>

        </behavior>

      </endpointBehaviors>

      <serviceBehaviors>                        

        <behavior name="DefaultBehavior">

          <serviceMetadata httpGetEnabled="true" />

          <serviceDebug includeExceptionDetailInFaults="false"/>

        </behavior>       

      </serviceBehaviors>

    </behaviors>  

    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" 

                               aspNetCompatibilityEnabled="true" />

  </system.serviceModel>

</configuration>

The Testing
We will just fast-forward through the hosting setup and service implementation.  Now that services are configured, built and running, we can perform the response testing and see how it all comes together.  When browsing to the webHttp base address and invoking the services via RESTful queries, I receive the following responses for the list and single respectively.

Then, to prove the JSON / XML switch, I used a fiddler software to create and review requests and responses.  First, I constructed an XML request and trapped the response.

Then, I constructed a JSON request and trapped the response.

As you can see, the service is responding to me in the request format.  For the client proxy implementation, review the source code attached at the end of the post.  The big victory here is the open and scalable approach to an SOA.  By doing dynamic communication in this manner, we are enabling a service to be consumed in the way that an integrator can best leverage.  This allows us to have a single implementation of intelligence and functionality while allowing any integrating software to choose the manner in which it interacts.  This is very powerful and very scalable and should allow your services to be consumer agnostic and focus strictly on intelligence.  For further review, download the entire project here.

Dynamic Types Using C#

I ran across a scenario the other day for creating a function to perform similar work on similar classes without having to be tightly coupled to any concrete type.  I found a few viable options and will be doing some examples of each to work through the exercise completely.  The first I want to talk through is the use of the dynamic keyword.

In short, using dynamic tells the compiler to ignore types at compile time and instead determine dispatch based on the actual type at run time.  Static binding of types does the exact opposite and performs a dispatch based on the concrete type.  Since code always illustrates these academic discussions more clearly, I created a simple project.  I have two examples of dynamic typing built into this single example.  I created some classes to illustrate a few types that are completely unrelated to each other. I also created a single class that was an example of type inheritance.

public class Account

{

    public String Name { get; set; }

    public Double Balance { get; set; }

    public Int16 AccountType { get; set; }

    public Account()

    {

        Name = "Account";

        Balance = 100.00;

    }

}

public class Customer

{

    public String Name { get; set; }

    public Double Balance { get; set; }

    public String CustomerType { get; set; }

    public Customer()

    {

        Name = "Customer";

        Balance = 900.00;

    }

}

public class Employee

{

    public String Name { get; set; }

    public Double Balance { get; set; }

    public String Department { get; set; }

    public Employee()

    {

        Name = "Employee";

        Balance = 500.00;

    }

}

public class Payable : Account

{

    public Payable()

    {

        Name = "Payable";

        Balance = -100;

    }

}

As you can see, the classes are completely independent of each other, but they have similar members.  Using the dynamic keyword, we can create a function that deals with these types at runtime and interacts with expected members at that time.

static void WriteDynamicObject( dynamic thing)

{

    Console.WriteLine("Name: "+ thing.Name + ", Balance: " + thing.Balance.ToString());

}

As long as the types I pass to this function have publicly accessible members named Name and Balance, this code will work with a complete disregard for compilation of any known type.  Next, to illustrate using the dynamic keyword for instructing the runtime to resolve the type regardless of declaration, I created overloaded functions that take a concrete type at each level of inheritance.

static void WriteConcrete(Account thing)

{

    Console.WriteLine("I am an Account Thing");

}

static void WriteConcrete(Payable thing)

{

    Console.WriteLine("I am a Payable Thing");

}

Finally, a simple console application that illustrates how these items work and are either statically or dynamically resolved.  The difference between the 'Concrete' functions is shown by the dynamic keyword telling the runtime to resolve this class at run time regardless of it's declaration.  Also, notice the last item is not even a class at all, but rather a dynamic type declared simply to be passed to this helper function.

static void Main(string[] args)

{

    // call writedynamicobject function for all concrete classes

    Account a = new Account();

    WriteDynamicObject(a);

    Employee e = new Employee();

    WriteDynamicObject(e);

    Customer c = new Customer();

    WriteDynamicObject(c);

    //call function for a concrete class, but resolve type at runtime

    Account p = new Payable();

    WriteConcrete(p);

    WriteConcrete((dynamic)p);

    // dynamically create values and pass as a dynamic type.

    WriteDynamicObject(new {Name="TotallyDynamicNonClass", Balance=250});

}

The output of this project illustrates how the runtime deals with each of these scenarios.

If you are like me, the first thought you have on using the dynamic typing pattern is along the lines of, "Couldn't you just use an interface or inheritance to accomplish the same thing?"  The answer is yes, assuming you are able to know those types at compile time.  But, there are also times where you can't know the definitions at compile time.  You can't compile the internet, but you can interface with it.  I would be remiss if I didn't get into the downsides of this practice as well.  This can open you up to having a program that is hard to maintain, difficult to debug and unit test and many similar issues. But, guns don't kill people, people with guns do.  Dynamic typing doesn't make code difficult to manage, a programmer does those things with his or her architecture.  This is not a magic bullet and should be used with great caution. The reason for using it is that some problems are inherently dynamic (e.g. web requests).  While you may use reflection for this sort of problem today, you may find dynamic typing a more expressive approach to the same problem.  Simply being dynamic in itself is a compelling argument. Reflection introduces a tightly coupled dependency on a static mechanism.  Dynamic methodologies are intent-based and trust the receiver to act upon the passed intent which creates a scalability unachievable by other means.  The source files are available for download here.