CRM Online to SharePoint Online Integration using REST and ADFS

Accessing SharePoint Online 2013 REST services with SSO via ADFS (Active Directory Federation Services) from CRM Online provides loads of potential opportunities, especially now that SharePoint offers a huge REST API. You can call REST from a CRM Online Plugin or Custom Workflow activity with no dependencies on SharePoint Client dll’s or Azure getting in the way, It is fairly awesome I must say.

I use the HttpWebRequest class to perform SOAP requests to perform the authentication part of this integration, which is totally supported within Sandboxed Plugins and Custom Workflow Activities. You can read more about the restrictions of the CRM Sandboxed environment here http://msdn.microsoft.com/en-us/library/gg334752.aspx. Once you get authenticated and obtain the cookies you are free to fire REST calls off by simply providing the cookies along with the request.

The best diagram I found to describe the authentication process visually was from Wictor Wilen’s blog http://www.wictorwilen.se/

Auth Process

I wanted to explore the issue of authentication between CRM Online and SharePoint Online from a server to server perspective and as you can imagine authentication is a big hurdle for this direct type of communication as every example I have seen so far includes Azure. Well at least until I started to look at what others were doing with Windows 8 Apps and Active Authentication. Omar Venado http://blogs.msdn.com/b/omarv/ and fellow MVP Wictor Wilen http://www.wictorwilen.se/ had some really great articles to help me build out a solution focused on this scenario. The code attached to this blog is heavily derived from Omar’s Windows 8 App code though now is useable in non Windows 8 App projects such as C# Console, SSIS Packages and of course CRM Plugin/Workflow projects. I have added some background reading at the bottom of this post so you can see the articles I used to get my code to work.

An example of how simple this code is to use

Uri spSite = new Uri(https://myOrg.sharepoint.com/sites/mySite”);

bool success = SpoAuthUtility.Create(spSite, username@domain.com, WebUtility.HtmlEncode(“Pword123”), false);

string odataQuery = “_api/web/lists”;

Uri url = new Uri(String.Format(“{0}/{1}”, SpoAuthUtility.Current.SiteUrl, odataQuery));

// Send a json odata request to SPO rest services to fetch all list items for the list.

byte[] result = HTTPHelper.SendODataJsonRequest(

url,

“GET”, // reading data from SP through the rest api usually uses the GET verb

null,

(HttpWebRequest)HttpWebRequest.Create(url),

SpoAuthUtility.Current // pass in the helper object that allows us to make authenticated calls to SPO rest services

);

string response = Encoding.UTF8.GetString(result, 0, result.Length);

You can authenticate using corporate credentials if inside your network, username/password if outside your network (e.g CRM Online O365) and you can also use your onmicrosoft.com accounts. Remember that my primary focus was on server to server integration possibilities, not user to server contexts such as in the current SharePoint CRM List Component model.

The attached C# Console app contains 3 files to provide you with an example of how to use the 2 cs classes HTTPHelper and SPOAuthUtility. To authenticate and start calling REST from CRM Plugins/Workflows simple include the HTTPHelper and SPOAuthUtility cs files into your Plugin/Workflow project.

Example App is available here https://skydrive.live.com/redir?resid=492A170E77E43399!830

Background reading

http://blogs.msdn.com/b/omarv/archive/2012/10/25/windows-8-store-apps-office-365-enterprise-preview-sharepoint-online.aspx

http://blogs.msdn.com/b/omarv/archive/2012/11/15/developing-windows-store-apps-for-sharepoint-online-with-sso-single-sign-on.aspx

http://www.wictorwilen.se/Post/How-to-do-active-authentication-to-Office-365-and-SharePoint-Online.aspx

http://allthatjs.com/2012/03/28/remote-authentication-in-sharepoint-online/

56 thoughts on “CRM Online to SharePoint Online Integration using REST and ADFS

  1. Pingback: Mahender Pal

  2. Vikram Singh

    Great Post Rhett ! thanks for sharing knowledge !!!

    I am trying this inside sandbox plugin but getting error “System.Security.SecurityException: That assembly does not allow partially trusted callers”. May be reason is HttpUtility is not supported in partial trust, not sure but got information from here

    http://social.msdn.microsoft.com/Forums/en-US/302377cf-97bd-4fe9-b7d0-fa7f08152927/systemsecuritysecurityexception-that-assembly-does-not-allow-partially-trusted-callers

    If I am trying this through console application or on-premise plugin(non-sandbox) then I am getting error “Given key was not present in the dictionary” in below line

    if (dynamicObject[“AuthURL”] != null)
    {
    corpAdfsProxyUrl = new Uri(Convert.ToString(dynamicObject[“AuthURL”]));
    }

    Please suggest, if I am missing something.

    thanks for your help !!!

    Reply
    1. Rhett Clinton MVP Post author

      Hi Vikram, I have only used the HttpUtility to HTMLEncode the password as I found when having characters like & in the password it would fail with Bad Request. I have found that WebUtility.HtmlEncode which is apart of System.Net works fine.

      Regards the Given key not found then it suggests the auth url is not found for you, this is the sts server. Inspect the return response and see if it is giving an error, you can view this in the code or by using Fiddler.

      Regards,
      Rhett

      Reply
      1. Vikram Singh

        Hi Rhett, Thank you for your help !!!

        #1, I have removed HttpUtility.HtmlEncode, now getting error “access security critical method ‘System.Web.Script.Serialization.JavaScriptSerializer..ctor()’ failed”

        #2, I tried to debug it and dynamic dynamicObject having only following Keys and values combination:

        State,4
        UserState,1
        Login,myuser@org.onmicrosoft.com
        NameSpaceType, Managed
        FederationBrandNme, ORG.ONMICROSOFT.COM

        Please help on this.

      2. Rhett Clinton MVP Post author

        Hi Vikram, I have updated the SPOAuthUtility class file as there was an issue in that O365 onmicrosoft accounts do not return a AuthUrl like federated ADFS accounts do when trying to retrieve the sts server address. This means the dynamic object will not be needed for your scenario so I added a simple check to make sure the json string contains the AuthURL value before serializing the string to dynamic.

        This should work for you now, just download the updated zip file.

        Cheers,
        Rhett

  3. Pingback: MSDynamicsWorld.com CRM Community Full Articles

  4. Pingback: Dynamics CRM field experts show off next gen integration; Can Microsoft keep up with the channel? - MSDynamicsWorld.com CRM Community Full Articles - Microsoft Dynamics CRM - Microsoft Dynamics Community

  5. Vikram Singh

    Wonderful, thank you so much Rhett ! It’s working now in console application.
    What about sandbox plugin?

    Reply
    1. Rhett Clinton MVP Post author

      Most definitely, I am providing a console main class to demonstrate its use but the exact same code works within a sandbox plugin. This was the main aim of this solution.

      Cheers, Rhett

      Reply
      1. Vikram Singh

        Hi Rhett, We are getting above #2 errror while using same code inside sandbox pluign. Have you tried to use same code inside sandbox plugin ?

  6. Pingback: CRM Online to SharePoint Online Integration using REST and ADFS - Mahender Pal - Microsoft Dynamics CRM - Microsoft Dynamics Community

  7. Dave Brockelsby MCT

    Great example Rhett! Saved me a lot ot time. Could you help me find the proper way to sent Update, Create and Delete POSTs via this code example?

    Reply
    1. Rhett Clinton MVP Post author

      Hi Dave, one of my most difficult things I’ve found is trying to get the REST services to function as the examples online seem to all be for JavaScript. I recommend looking for other blogs that have covered examples of this already as I do not have thorough knowledge of working with the full SharePoint REST interface yet.

      Cheers,
      Rhett

      Reply
  8. mydevexperience

    Hi Rhett,

    I did tried to used your code in a console application and it worked very well but using same code inside sandbox plugin fails with the following error message

    Assembly ‘System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ is a conditionally APTCA assembly which is not enabled in the current AppDomain. To enable this assembly to be used by partial trust or security transparent code, please add assembly name ‘System.Web.Extensions, public key = xxxxx to the PartialTrustVisibleAssemblies list when creating the AppDomain.

    Any idea why it failing for CRM online? Any help will be really appreciated.

    Reply
      1. mydevexperience

        Hi Rhett,

        Thanks for your kind reply. Your latest code already has WebUtility.HtmlEncode and that’s the code I used earlier. Do you need any other assemblies for the build? I mean using ILMerge. Please let me know. Have a good day.

        Thanks a lot

  9. mydevexperience

    Hi Rhett,

    Thanks a l lot for getting back to me. I did tried in a console application with partial trust and I got the following error.

    Assembly ‘TestConsoleAppSPRest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null’ is marked with the AllowPartiallyTrustedCallersAttribute, and uses the level 2 security transparency model. Level 2 transparency causes all methods in AllowPartiallyTrustedCallers assemblies to become security transparent by default, which may be the cause of this exception.\n\nAssembly ‘System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ is a conditionally APTCA assembly which is not enabled in the current AppDomain. To enable this assembly to be used by partial trust or security transparent code, please add assembly name ‘System.Web.Extensions, PublicKey=xxxxxxxxx’ to the the PartialTrustVisibleAssemblies list when creating the AppDomain.”

    This assembly is marked as conditionally APTCA (allow partially trusted callers attribute), which means that code running in the Sandbox won’t be able to call into it without product changes to CRM. Without this change, there isn’t a supported workaround that I am aware of.

    If you have any other thoughts please share it.

    Thanks a lot

    Reply
    1. Rhett Clinton MVP Post author

      The javascript serializer was used in a later version of this utility but now I use Newtonsoft to handle json responses. Prior to this I just just used the xmldocument or similar to handle the response but I’m having trouble trying to find the original version of my project.

      If you have time you could just remove the reference and handle the response without it. It is used only once I think.

      Otherwise I will try to get this updated soon.

      Cheers, Rhett

      Reply
  10. mydevexperience

    Hi Rhett,

    Tried the Newtonsoft and the console app works fine but CRM complains about Newtonsoft component missing error. Looks like I need to use ILMerge which probably not a supported way for Microsoft. I will let you know how this goes. Is any other way you could think about avoiding ILMerge?

    Cheers

    Reply
    1. mydevexperience

      Hi Rhett,

      We raised a support call with Microsoft and suggested that to host a web service in Windows Azure which acts as an interim between SP 2013 online and CRM 2011 online. This will help in synching the data between the two applications. So that means no other options to automate this from a plugin sandbox mode 😦

      Reply
      1. Aravindan Muniappan

        Hi Rhett, I have used your code for sharepoint integration. Thank You for sharing us the code. Instead of using JavaScript Serializer, I have used DataContractJsonSerializer to deserialize JSON response. Below is the code snapshot.

        // convert string to stream
        byte[] byteArray = Encoding.UTF8.GetBytes(Encoding.UTF8.GetString(result, 0, result.Length));
        //byte[] byteArray = Encoding.ASCII.GetBytes(contents);
        MemoryStream stream = new MemoryStream(byteArray);

        //MemoryStream stream1 = new MemoryStream();
        DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(ContextInformation));
        stream.Position = 0;
        ContextInformation ci = (ContextInformation)ser.ReadObject(stream);
        digest = (ci != null) ? ci.d.GetContextWebInformation.FormDigestValue : String.Empty;

        Thanks,
        Aravind

      2. Rhett Clinton MVP Post author

        That’s great Aravind, thanks for sharing. When I used that I didn’t like that some objects had d as the parent object so I found it inconsistent when trying to make a generic style function. But it works well if you code for specific scenarios as you have done with your example.

        Many thanks,
        Rhett

  11. Pingback: Dynamics CRM & SharePoint Integration Overview | Dynamics CRM Chat

  12. Choudhury

    I too had the same issues when trying to deploy the plugin in sandbox mode. Removing references of System.Web fixed those issues. Can someone please tell what changes are required in the ContextInfo class ?
    The reply from Aravindan says that digest header can be obtained as shown below:
    digest = (ci != null) ? ci.d.GetContextWebInformation.FormDigestValue : String.Empty;

    However, can you please let me know how to retrieve GetContextWebInformation from the d property ?

    Reply
  13. Pingback: Dynamics CRM Chat – Staying on top of CRM 2013 Certification!

  14. Simon

    Hi Rhett, I’m trying to POST to create a folder in a list – but I get a 403 Forbidden. I’m new to SharePoint dev so I’m not sure if I just have to set a permission somewhere?

    I’m using an O365 E3 trial , and your console app example. The username and password I’m using are of the O365 administrator.

    I’ve changed the SendODataJsonRequest to be a “POST” and set the requestContent parameter to “new byte[0]”. The query I’m attempting looks like this; restQuery = “_api/Web/GetFolderByServerRelativeUrl(‘/Request/2014’)/Folders/add(‘Test’)”;

    There exists a folder structure “Request/2014” and I can GET information about this folder… It’s the /Folders/add that is failing.

    Any ideas or pointers?
    Thanks!

    Reply
    1. Simon

      With further investigation I discovered my error – I needed to pass the X-RequestDigest header in the POST for authentication, so I simply added the following code as the final parameter to the SendODataJsonRequest method;

      new Dictionary {
      { “X-RequestDigest”, SpoAuthUtility.GetRequestDigest() }
      }

      Reply
      1. Hiran Amarasinghe

        Hi simon, I also experience the same issue when tried to make a POST request. Even though I added the X-RequestDigest header in the POST, still I get the 403. Can you kindly advice me on how you overcame this issue. Thanks alot

  15. Arnaud

    Hi Rhett,
    thank you so much for sharing this. I used your code in combination with Aravind suggestion for serialization. It works perfectly in sandbox mode so far.

    cheers,

    Arnaud

    Reply
  16. Pingback: Scott Durow's Dynamics CRM Blog

  17. Bob

    Can I ask you which cookies you are talking about when you say “Once you get authenticated and obtain the cookies you are free to fire REST calls off by simply providing the cookies along with the request.” ?

    I am using Java to authenticate against our ADFS server using SOAP. I only get 1 cookie back, ReqClientId. However, when I do a normal browser login to Dynamics I am seeing 4 cookies: MSISAuth, MSISAuth1 ReqClientId and MSISSignOut.

    Anyway, when I try to call the oData REST endpoint, passing along the ReqClientId cookie, I get the login page, meaning I am not authenticated.

    I was just curious if you could tell me what cookies *you* are getting back.

    Thanks!

    Reply
  18. anwar khan

    Hi all ;

    i am trying to create a site when i add account in crm means for each account we create indivudual site and then in that site i want to create a folder with the account name can any one suggest how can i achive this i am getting connected to shrepoint successfully

    Reply
  19. anwar khan

    Hi i am getting the following error on this line

    SpoAuthUtility.GetRequestDigest()

    cannot be serialized. Consider marking it with the DataContractAttribute attribute, and marking all of its members you want serialized with the DataMemberAttribute attribute. If the type is a collection, consider marking it with the CollectionDataContractAttribute. See the Microsoft .NET Framework documentation for other supported types

    Reply
  20. Simon

    Hi.

    I’m trying to ge this work on CRM 2015 Online but get an error from the SharePointMethods constructor. It’s calling GetCookieContainer method, which calls GetSPOAuthCookies, and that is where the error is thrown.
    It says “The remote server returned an error: (411) Length Required”.

    Why is this happening?

    Reply
  21. danutonline

    Thanks a lot for all your work and especially for sharing it.

    There is very little documentation online regarding this topic and your post helped me a lot.

    Reply
  22. Ketan

    Hello Rhett,
    Great Article thank you for sharing it.
    I wanted to know what if you were trying to do the same stuff from CRM on-prem to SharePoint on-prem would this still work?
    Also the reason for doing this is wanted to create such a utility using SharePoint REST calls and call them from CRM (on-prem) and Later on when we move to CRM online just port the same plug-in model to CRM online plugin without chaning the SharePoint code. Since only thing changing would be SP-on prem to SharePoint Online (SPO). Can this code work for both on-prem SharePoint and SPO (i.e. the REST calls, authentication would differ)?

    Appreciate your response.
    Thank you.

    Reply
  23. Chaussade

    Hi,

    I am working on a Dynamics CRM online workflow. I wish to update the sharepoint folders attributes in the code. The Sharepoint online is in the same tenant.

    Can I still use this way to manage the folder items in sharepoint from a CRM workflow?

    Regards,
    Baptiste

    Reply
  24. clc136a

    I’m interested in enhancing the authentication class to handle on-premise scenarios without having to add SharePoint DLLs to the GAC and running non-isolated. The OOTB integration using high-trust works OK but not enough control. Do you have any suggestions for getting an access token to use in the REST service? Thanks, Chris

    Reply
      1. clc136a

        Hi Rhett, had a look but no joy. Using NTLM and credentials I can authenticate but not getting cookies or soap response. I think I need this to authenticate REST calls? There is a hidden digest field though… <input type="hidden" name="__REQUESTDIGEST" id="__REQUESTDIGEST" value="0x949A2478815D73… I also found this http://www.sharepointblogs.be/blogs/vandest/archive/2014/09/12/calling-web-services-in-nintex-workflow-and-different-authentication-mechanisms.aspx
        Kind regards, Chris

  25. Debajit

    Hi Rhett,

    Wonderful post and I can’t tell you how much this has helped me. With your code, I was successfully able to create a folder and file in sharepoint from CRM Online. However I am stuck at one point. I need to assign permissions to users to a folder in sharepoint using the REST based stuff. I am completely struck on this as I dont know how to form a valid query for this. Your expert help is much appreciated.

    Regards
    Debajit

    Reply
    1. Dennis94

      Hi Debajit,

      I am trying to do the same like you (create folders/files), but i get a 400 invalid request. Do you have any help for me or could you share your code?

      Regards
      Dennis

      Reply
  26. Robert Seiffert

    Hi Rhett,

    I am currently trying to use your code in my CRM Online Plugin on create of Account to access a SharePoint Online site within the same Office365 suite but get an error on this.adfsAuthUrl = GetAdfsAuthUrl() in the GetMsoStsSAMLToken in the SPOAuthUtility class. The error is sadly not very informational because it only says: System.Security.SecurityException: Error in the request.
    Is there anything that pops into your mind what could cause this problem?

    Greetings from Germany!

    Reply
  27. Mark Pitts

    Hi Rhett,

    Would you, per chance, know how to implement ‘act-on-behalf’ impersonation? My code will originate from a Dynamics 365 plug-in, where I want the file, which my plug-in code uploads to SharePoint, to be owned by the plug-in’s user context.

    Thanks,
    Mark

    Reply

Leave a reply to Vikram Singh Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.