Apologize for the unfinished blog post

I accidentally published an unfinished draft blog post today, it was something in my queue that I have been trying to finish, but was not able to dedicate time to it. I published it by mistake after I clicked a wrong button.

I sincerely apologize for the confusions that it may have caused. I will try to make the post(s) available as soon as I can.

[UPDATE - May 10, 2012] The series of blog posts is available now, it is still currently working-in-progress.

The other half story about BDD, SSIS, and CRM

[UPDATE - July 7, 2017] In our most recent v9 release, we have added the support of multi-threaded writing in our CRM destination component, so this blog post is no longer useful. You can simply turn on multi-threaded writing in the destination component to achieve the same (with much greater flexibility or even better performance). [/UPDATE]

[NOTE - May 5, 2016] The config file changes are no longer necessary since our v7.1 release on May 5, 2016. The technique described in this blog post is still useful if you are using an older version or even if you may be using a different approach for data migration or integration.

This is an update of my previous blog post about using BDD component in SSIS to improve the performance of loading data into Microsoft Dynamics CRM.

I was only able to tell the half story of the technique in the previous blog post. Here is the other half.

In order to make full use of the BDD component in SSIS data flow, you need to increase the connection limit that is imposed by Microsoft .NET framework, which is a maximum of 2 connections per host (e.g. server). In order to overwrite this limit, you need to modify DTExec.exe.config and DtsDebugHost.exe.config files under DTS\binn folder by adding the following connectionManagement section.

<configuration>
 ...
 <system.net>
   <connectionManagement>
     <add address="*" maxconnection="100"/>
   </connectionManagement>
 </system.net>
</configuration>

The above configuration allows up to 100 connections per host at the same time. You may change the number based on your needs. Note that if you are using 64-bit system, you will need to make changes to the files under both Program Files and Program Files (x86) folders. 

After making the above changes, I observed more performance improvement in my SSIS data flow. I was able to load 0.9 millions of records into CRM within one hour (5 outputs and 10 outputs had almost identical performance benchmark, while 5 outputs outperformed 10 outputs a bit in my test). 

This has been reflected in our product FAQ page. 

If you write custom code to load data into CRM using multi-threading, you should make similar changes to your application's config file as well. 

It should also be noted, if your SSIS data flow reads or writes data simultaneously to a single host using web service interface, you should consider making this change, regardless it is WCF, WSDL service, or maybe even a REST service. 

This is a tip that I learned from CRM project manager Mahesh Hariharan at xRM Acceleration Lab during the week of Apr 2 to Apr 6, which was held at Microsoft campus in Redmond. 

BTW, in case you don't know, Microsoft has an xRM Acceleration Lab program which is open to Microsoft Dynamics CRM partners to help them build up the necessary technical skills to bring your solutions faster to the market. The lab is a one-week one at Microsoft campus, which consists of a number of technical sessions presented by CRM MVP folks and some product team members. The best part about the lab is, your team resources will be working with a gang of MVP folks (and some product team members) side-by-side to develop solutions during the week. This is a very efficient way to build up the technical skills by building something real on top of Microsoft Dynamics CRM platform. 

Hope this helps. 

Release: Service Pack 3 of SSIS Integration Toolkit for Microsoft Dynamics CRM

 Today, we are happy to announce the third service pack of SSIS Integration Toolkit for Microsoft Dynamics CRM, which is now available for download. The release includes the following updates and fixes.

• Update: We made a minor change to License Manager program, which now prohibits possible duplicate license requests by clicking the OK button more than once in "Enter Product License" window
• Fixed: In CRM Source Component Editor, when you enter a valid FetchXML, and immediately click "OK" button to save the change, the source component does not properly populate CRM fields due to the lazy-editing feature (for performance reason) used in the XML editor component
• Fixed: When you update statecode and statuscode for salesorder or opportunity entity, you might run into exceptions
• Fixed: The CRM destination component doesn't provide correct CRM record's ID in GUID value after the record has been created in CRM (Thanks to Javier for reporting this issue)

A new build is now available for download at our product download page.

In order to install the new build, you will need to uninstall the previous version first.

On a side note, from this post on, I will move such kind of product announcements to the blog section of my company website. But I might share some product news here if it is something technically interesting.

Thank you for reading.

Why should you use CRM Developer Toolkit for your next CRM project engagment

[DISCLAIMER] This blog post contains strongly-opinionated information. For this reason, you are expected to use your judgement to decide whether the information is helpful for your development practice. As such, I make no warranties of any kind. [/DISCLAIMER]

Many of you are probably aware that I am a big fan of CRM Developer Toolkit. However, it is an unfortunate fact that the tool has not attracted enough attention in the community. Many of us are sceptical about using the toolkit with the concerns that the complexity or limitation of the tool might overweigh its benefits. This is reasonable for a new tool, and it is also expected.

With my limited experience with the toolkit, I have identified quite some important benefits when the tool is properly used. Here are some of them.

Unified Development Experience

The toolkit helps handle most day-to-day CRM development tasks. The toolkit supports the development of CRM web resources, Plugin, Silverlight, Workflow projects.

The toolkit provides a consistent approach for CRM development practice. Using the toolkit, you have one single Visual Studio solution to work with, which allows you to manage most (if not all) of CRM artifacts. The solution file include a CrmPackage project which is the container of CRM artifacts that you deploy and publish to CRM, including the source code of web resource files, references to plugin, workflow, Silverlight projects. If any of us have worked with SharePoint, you probably remember the days before its Visual Studio extension was available. If you asked 10 developers to do one single task for SharePoint, in most of cases, they would probably have done it in 9 different ways. It has been significantly better after the SharePoint extension became available. I believe CRM developer toolkit is an equivalent tool of the Visual Studio extension in SharePoint development practice, which has been widely embraced and welcomed by SharePoint community, and it has become part of Visual Studio shipment in VS 2010. I dare to say that nobody in SharePoint community wants to go back to the time without the extension. I would hope that the CRM developer toolkit can one day make into Visual Studio shipment as well, however I am very aware that we have a long way to go to get there.

Increased Development Productivity

The toolkit significantly improves your development productivity. Working with CRM web resources can be a very tedious task if you do it from CRM user interface. The process is significantly simplified, if you are using the toolkit. You make the changes to the web resource within Visual Studio, then you right-click CrmPackage project, and choose "Deploy" menu, the websource is deployed to CRM. It is as simple as a snap. The only thing you need to do after this is, you have to publish the changes in order to see the changes in effect. What I usually do is to always leave a web page open, which has a "Publish All Customizations" button on it, so that it takes me one single click to have the deployed changes published in CRM. Since the toolkit supports most CRM development tasks, you rarely need to leave the environment. Also the toolkit has included a few other productivity tools including the capability to create a plugin class, and generate early bound entity classes from CRM Explorer, etc.

Managed Development Process

The toolkit manages your CRM web resources so that you can source control them. The toolkit allows you to manage your CRM web resources using file system within Visual Studio, each web resource is represented by one physical file in CrmPackage project. Such files can be easily version controlled using your favorite source control system. In the good CRM4 days, I have used some code snippet to load JavaScript from file system, so that I can version control my JavaScript files. This is totally unnecessary with the help of the develop toolkit in CRM2011 (also thanks to the Web Resource feature).

Advocate of Best Practices

The toolkit provides some guidelines for CRM development practice, and also it advocates some best practices when working with CRM. It has got a plugin project template in the Visual Studio solution for us to start with, which includes quite some informative comments.

Free of Charge

Last, but not the least, the toolkit is free, it comes with CRM SDK at no charge. ;-)


With all the above benefits using the toolkit, you should also be mindful about the constraints or limitations when using the tool. It is particularly important to know the boundary of the tool, and know what the tool can do, what it can't do or should  not be used. With the help of the toolkit, we, the developers, are really the director of the development. The tool has its own way to get the job done, it is up to us to leverage its flexibility to make our development life more enjoyable. After all, CRM is the final judge, the toolkit won't cover 100% of development scenarios, but there is usually some workaround when it comes to CRM platform, the primary value of the tool is to help streamline the process of working with CRM, so that we can be more productive.  Some in the community may say that the toolkit is just a CRM metadata browsing tool. This could be the case for the toolkit's CRM4 version, and it is definitely not the case for CRM 2011 developer toolkit, it is much more than a CRM metadata tool.

Although the toolkit is overall a great tool, it doesn't mean it is perfect. One important feature that is currently missing in the toolkit is the ability to deploy individual WebResource (CRM MVP Rhett Clinton and me both raised this as a feature request). I think this is an important feature to make the toolkit work better in a shared team development environment. For instance, if we have two developers using the same DEV environment, both of them are working on a different piece of web resource at a time. With the current version of toolkit, it will deploy the entire project when the Deploy command is used, so what would typically happen is, one developer will overwrite another developer's latest change. If we have the capability to deploy individual file, it will not cause this kind of trouble. Please vote for the change if you agree with me, so let the voice heard. Also there are also other areas that could make the toolkit better, if you have any other feature requests, I would encourage you to use Microsoft connect site to report them to Microsoft.

Thanks for reading.