On Wednesday John Papa will be giving a webcast about anything and everything you ever wanted to know when consuming Astoria Data Services from a Silverlight 2.0 app.  Should be a good one. I also noticed that his new book will be out January 2nd.  I have already preordered my copy.  I do wonder where O'Reilly is finding animals for the covers these days.

Over the past few weeks I have posted a couple of samples of implementing IUpdateable for non-EF DALs so those can be used as data sources for ADO.NET Data Services.  A handful of people have asked me why this is interesting, so I now realize I should have given more background up front. Out of the box, the only DAL that gets read/ write behavior for free in ADO.NET Data Services is Entity Framework.  This is because internally in the ADO.NET Data Services bits, there is an implementation of an interface we call IUpdatable: namespace System.Data.Services { public interface IUpdatable { void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded); void ClearChanges(); object CreateResource(string containerName, string fullTypeName); void DeleteResource(object targetResource); object GetResource(IQueryable query, string fullTypeName); object GetValue(object targetResource, string propertyName); void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved); object ResetResource(object resource); object ResolveResource(object resource); void SaveChanges(); void SetReference(object targetResource, string propertyName, object propertyValue); void SetValue(object targetResource, string propertyName, object propertyValue); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The reason behind this interface is that when we were designing V1 of ADO.NET Data Services we wanted to the user to have the ability to plug in LINQ data sources(.e. Linq to Sql, Entity Framework, nHibernate, etc).  For read only access, this is really straight forward and general uniform across LINQ DALs.  All the data source has to expose is a type with public IQueryable<T> entry points for each of their EntitySets.  For write operations, it was a completely different story since there is no common API for updating LINQ based DALs.  So we invented IUpdateable. So any data source for Astoria that wants to support PUT, POST, and DELETE they must have an implementation of IUpdateable for their specific DAL. Hence why I started projects on Code Gallery to provide reference implementations of implementing IUpdatable for SubSonic and Linq to Sql. There actually are a few more smaller interfaces to implement to become a full fledge ADO.NET Data Service provider.  At some point in the near future, I will try to get a post up describing that stuff plus some preview of the refactoring we are doing in V2 of ADO.NET Data Services to make a nicer provider plugin model.

One of the most common feature requests we hear for Astoria is the ability to mark properties on an entity as hidden.  Currently in V1 we support this on the server, but not with the .NET or Silverlight clients. Phani, resident Astoria hacker (and a really good tester), has written a very good post on how you can roll your own override of client serialization behavior to support this functionality with the v1 clients.

Finally got IUpdateable.ResetResource() for Linq To Sql working correctly.  This means that REPLACE operations are now working correctly.  I have uploaded those changes to code gallery. Going forward all changes to the code with be available exclusively on code gallery.  Also, I am trying to track known open issues there with the issue tracking capability. Finally, if anyone wants to be a contributor to this project or the SubSonic one, please let me know.  Particularly for the latter, there is some considerable work to do.

I have developed a data provider for SubSonic to enable it as a ADO.NET Data Services data source. This ended up being a bit harder then I initially thought, and is definitely still a work in progress. If you are interested, I have started a code gallery project for it. I decided to base the first implementation of my provider on SubSonic 3.0 preview 2.  This is the first version of Sub Sonic that supports IQueryable<T> so getting read-only ADO.NET Data Service support over SubSonic is quite trivial. In fact, Jay Kimble has written a very nice post about how to do it here. To use my SubSonic Astoria data provider, first follow Jay’s instructions to set up your SubSonic DAL.  This includs a very nice addition he has made to the SubSonic T4 templates to automatically add the Astoria attribute for Key properties. Next, down load the file SubSonicIUpdateable.cs class from the code gallery project. Add it into your project and make it a partial class with the same type as the DB type generated by SubSonic code gen: public partial class DB : IUpdatable .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Interestingly, SubSonic (at least version 3.0) doesn’t really have a traditional O/R context type per se. In a way, this is practically a requirement for being a Astoria data source. Hence, my implementation utilizes a local context for each set of requests. Since this is only required to support scenarios required by IUpdateable, this was rather trivial to implement. A few known bugs/ issues with the initial code: Binary fields support is currently not working. Hence inserting/ updating binary data is not happening. I think this is a SubSonic bug, but I need to spend some more cycles looking into it. The IUpdateable implementation currently does not support batch mode. So even if the Astoria client is specifying batching, the DAL will send in the changes one at a time. Only using SubSonic metadata to find primary key properties for a given type for concurrency checks. Probably should switch this to using Astoria metadata for this. Identity values not working. I can’t currently figure out how to get this to work with SubSonic 3. I currently don’t support relationships between entity types because that is not currently supported with the preview of SubSonic. I will try to keep this up to date. Also, along with the code I have included the test project I use to test the implementation. One other interesting design issue that I discovered while implementing this is how Astoria Data Providers must be integrated at the source level. For example, what I really would like to do is implement something like SubSonicAstoriaProvidr<T> (where T is the DAL instance) and ship a assembly for that. With Astoria V1, that is next to impossible so one has to utilize something like partial classes to make this work. More to come on this later, but this has become one of the most important things we want to improve on in vNext.

As promised the slides & demos for my two ASP.NET Connections talks from 11/11/08:       MDA01: Deep Dive: ADO.NET Data Services Framework—Application Patterns       MDA04: LINQ to XML, SQL, Entities, DataSets and Co.: Data Access Technologies Explained  - AKA Linq to Anything   Update: 11/12/08 - download now includes all demos including Linq to Twitter, PetBook, and my LINQ samples from our LINQ to Anything (MDA04) talk.  Will update again with Shyam's demos from LINQ to Anything sometime in the next few days.

IUpdatable for Linq to Sql I have had an implementation of IUpdateable for Linq to Sql about half done for quite some time now. Yesterday I decided to bite the bullet and finish it up. Below I have included that code. I’ve done some light testing, but I am sure there are some bugs – and perhaps some perf fixes that can be made. My plan is to get this out on CodePlex as soon as I get a moment.  Update - 11/06/08, I actually ended up putting it on CodeGallery here. Update 2 - 12/02/08, I found I had implemented ResetResource incorrectly to set the property values to the original values instead of the default values.  Fixed this and uploaded the file to CodeGallery. Update 3 - 12/11/08 - Found another set of fixes for ResetResource.  Fixed this and uploaded the file to CodeGallery Note: Going forward all updates to the code will be exclusively on Code Gallery and will not posted to this blog. A couple of notes about this code: 1) I haven’t implemented ClearChanges() yet. As far as I can tell there is really no straight forward to do this with Linq to Sql. It isn’t that big of deal because ClearChanges is only required by Astoria when processing batches with multiple ChangeSets. Something that is not supported by the Astoria .NET client. 2) The easiest way to use this is to define a partial class for your DataContext type and provide the implementation there. i.e. public partial class nwDataContext : IUpdatable That way you don’t need to change your generated context class. Next step - implementing IExpandProvider public partial class myDataContext : IUpdatable { /// <summary> /// Creates the resource of the given type and belonging to the given container /// </summary> /// <param name="containerName">container name to which the resource needs to be added</param> /// <param name="fullTypeName">full type name i.e. Namespace qualified type name of the resource</param> /// <returns>object representing a resource of given type and belonging to the given container</returns> public object CreateResource(string containerName, string fullTypeName) { Type t = Type.GetType(fullTypeName); Debug.Assert(t != null); // assume can find type ITable table = (ITable)this.GetType().GetProperty(containerName).GetValue(this, null); object resource = Activator.CreateInstance(t); table.InsertOnSubmit(resource); return resource; } /// <summary> /// Gets the resource of the given type that the query points to /// </summary> /// <param name="query">query pointing to a particular resource</param> /// <param name="fullTypeName">full type name i.e. Namespace qualified type name of the resource</param> /// <returns>object representing a resource of given type and as referenced by the query</returns> public object GetResource(IQueryable query, string fullTypeName) { object resource = null; foreach (object o in query) { if (resource != null) { throw new Exception("Expected a single response"); } resource = o; } // fullTypeName can be null for deletes if (fullTypeName != null && resource.GetType() != Type.GetType(fullTypeName)) throw new Exception("Unexpected type for resource"); return resource; } /// <summary> /// Resets the value of the given resource to its default value /// </summary> /// <param name="resource">resource whose value needs to be reset</param> /// <returns>same resource with its value reset</returns> public object ResetResource(object resource) { Type t = resource.GetType(); Debug.Assert(t != null); object newResource = Activator.CreateInstance(t); MetaTable table = this.Mapping.GetTable(t); foreach (var member in table.RowType.IdentityMembers) { object keyValue = member.MemberAccessor.GetBoxedValue(resource); member.MemberAccessor.SetBoxedValue(ref newResource, keyValue); } return newResource; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } /// <summary> /// Sets the value of the given property on the target object /// </summary> /// <param name="targetResource">target object which defines the property</param> /// <param name="propertyName">name of the property whose value needs to be updated</param> /// <param name="propertyValue">value of the property</param> public void SetValue(object targetResource, string propertyName, object propertyValue) { PropertyInfo pi = targetResource.GetType().GetProperty(propertyName); if (pi == null) throw new Exception("Can't find property"); pi.SetValue(targetResource, propertyValue, null); } /// <summary> /// Gets the value of the given property on the target object /// </summary> /// <param name="targetResource">target object which defines the property</param> /// <param name="propertyName">name of the property whose value needs to be updated</param> /// <returns>the value of the property for the given target resource</returns> public object GetValue(object targetResource, string propertyName) { PropertyInfo pi = targetResource.GetType().GetProperty(propertyName); if (pi == null) throw new Exception("Can't find property"); return pi.GetValue(targetResource, null); } /// <summary> /// Sets the value of the given reference property on the target object /// </summary> /// <param name="targetResource">target object which defines the property</param> /// <param name="propertyName">name of the property whose value needs to be updated</param> /// <param name="propertyValue">value of the property</param> public void SetReference(object targetResource, string propertyName, object propertyValue) { this.SetValue(targetResource, propertyName, propertyValue); } /// <summary> /// Adds the given value to the collection /// </summary> /// <param name="targetResource">target object which defines the property</param> /// <param name="propertyName">name of the property whose value needs to be updated</param> /// <param name="resourceToBeAdded">value of the property which needs to be added</param> public void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded) { PropertyInfo pi = targetResource.GetType().GetProperty(propertyName); if (pi == null) throw new Exception("Can't find property"); IList collection = (IList) pi.GetValue(targetResource, null); collection.Add(resourceToBeAdded); } /// <summary> /// Removes the given value from the collection /// </summary> /// <param name="targetResource">target object which defines the property</param> /// <param name="propertyName">name of the property whose value needs to be updated</param> /// <param name="resourceToBeRemoved">value of the property which needs to be removed</param> public void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved) { PropertyInfo pi = targetResource.GetType().GetProperty(propertyName); if (pi == null) throw new Exception("Can't find property"); IList collection = (IList)pi.GetValue(targetResource, null); collection.Remove(resourceToBeRemoved); } /// <summary> /// Delete the given resource /// </summary> /// <param name="targetResource">resource that needs to be deleted</param> public void DeleteResource(object targetResource) { ITable table = this.GetTable(targetResource.GetType()); table.DeleteOnSubmit(targetResource); } /// <summary> /// Saves all the pending changes made till now /// </summary> public void SaveChanges() { this.SubmitChanges(); } /// <summary> /// Returns the actual instance of the resource represented by the given resource object /// </summary> /// <param name="resource">object representing the resource whose instance needs to be fetched</param> /// <returns>The actual instance of the resource represented by the given resource object</returns> public object ResolveResource(object resource) { return resource; } /// <summary> /// Revert all the pending changes. /// </summary> public void ClearChanges() { } } .csharpcode { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff } .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { margin: 0em; width: 100%; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 }

Along with my manager Shyam Pather, I am giving the following talk at the upcoming Dev Connections conference:   MDA04: LINQ to XML, SQL, Entities, DataSets and Co.: Data Access Technologies ExplainedWith the release of the Microsoft .NET Framework 3.5, Microsoft introduced several new data access technologies based on its LINQ technology, such as LINQ to SQL and LINQ to DataSets. Shortly after, we expect Microsoft to release the ADO.NET Entity Framework which enables LINQ to Entities in addition to Entity SQL. Finally, we have all the existing ADO.NET data access patterns. This raises one important question: Which data access technology is best suited for which situation? In this session, we look at different scenarios and requirements that can inform your technology decision.   As part of this talk, we are trying to give as many Linq demos as possible.  So far we have:   Linq to Objects Linq to Sql Linq to Entities Linq to DataSet Linq to Xml Linq to ADO.NET Data Services Linq to Windows Azure Storage Linq to Sql Data Services   Plus a few surprises.    I am looking at something real esoteric to write a Linq provider.  So if anyone has any ideas, please leave a comment or email me.

The Astoria team is hiring! We have positions open for Developers and Program Managers.  Currently, we are finishing up V1 and planning for V2, so it is a great time to join the team.  For V2, we are thinking about expanding the project into new areas (e.g. offline support, sync, dynamic language support) so there will be plenty of new and exciting work. Please forward your resume to aconrad@microsoft.com if you’re interested.  The project covers a diverse set of technical areas (databases, Linq, O/R, REST, etc) so we value passion for the technology more than deep expertise in any particular area. The Astoria team uses an agile development model, so the project moves fast and we encourage individuals to take on a variety of roles and responsibilities. If you’re a developer who wants to be responsible for more than just coding (i.e. API design, specification writing, working with customers and partners) this team is the place for you! And finally, team members must be willing to participate in the grueling Team Astoria off-site events. Update 4/23/08 - As of the start of this week, we now have a block of additional new positions on the Astoria Team we need to fill.  These include development, program management, and quality assurance positions.  We are trying to fill these very quickly, so if you are interested please contact us soon.

The news is out! At Mix, Windows Live will be announcing several new Live Services with AtomPub endpoints.  These services will also be compatible with Astoria and therefor will work with the Astoria client library.  An excerpt from David Treadwell's post: Microsoft is making a large investment in unifying our developer platform protocols for services on the open, standards-based Atom format (RFC 4287) and the Atom Publishing Protocol (RFC 5023). At MIX we are enabling several new Live services with AtomPub endpoints which enable any HTTP-aware application to easily consume Atom feeds of photos and for unstructured application storage (see below for more details). Or you can use any Atom-aware public tools or libraries, such as .NET WCF Syndication to read or write these cloud service-based feeds. In addition, these same protocols and the same services are now ADO.NET Data Services (formerly known as “ Project Astoria”) compatible. This means we now support LINQ queries from .NET code directly against our service endpoints, leveraging a large amount of existing knowledge and tooling shared with on-premise SQL deployments. The intent for these early, experimental releases are to gather valuable feedback from the community around our idiomatic and freely licensed extensions to AtomPub which deal with important service scenarios, such as URL formats, nested directories, image streams, and service metadata. You can read more about this on the Project Astoria team blog. There will also be a few more surprises in this area announced at MIX. Stay tuned. Try them out and give us your feedback! At Mix, we have a Project Astoria talk all about using the Astoria client library to consume Live services.  RESTful Data Services with the ADO.NET Data Services Framework Wednesday, March 5 4:30 PM - 5:45 PM, Lando 4204 Speaker(s): Pablo Castro Audience(s): Technical Session Type: Breakout Learn how to use ADO.NET Data Services Framework to easily create and consume REST data services on the web. This session will cover the main concepts of the ADO.NET Data Services Framework (aka Project "Astoria"), show how to use it, and discuss how to use it with Microsoft's broader vision of a common interface for Windows Live and 3rd party services. So if you are going to Mix, be sure to go to that talk.  The Mix talks are also available online 24-48 hours after they are given. On a related note Pablo, Mike, and myself will all be at Mix next week.  We should be spending a lot of time in the hands on lab/ Open Spaces area.  If anyone has any questions about any of our projects, wants to see some Astoria demos, etc - drop  me a line and we can set something up.

Last Wednesday we had a very important offsite for the entire Project Astoria team: Many key technical issues were discussed. In fact, the entire team was quite spent after a long, hard day. 

Michael Sync has written a sample using the new Astoria Silverlight client library. David Hayden posted a nice walk through for getting an Data Service up and running over Linq To Sql. Our own Macelo Ruiz talks about some tweaks we made to the Astoria URI syntax on his personal blog.

Matthew Wills ported my original C# CopyToDataTable<T> Linq operator to VB.  According to him, he used Instant VB to convert the code from C# to VB and then hand tweaked it.  Looks like a fairly cool tool that we could use to port our C# unit tests to VB.   (We try to run the same functionals in both VB and C# these days because of the differences in Linq and comprehension support between the two compilers).

The updated Astoria Silverlight client is now available. This client is 95% compatible with the client library we released back in the December CTP (we hope to make this 100% source code compatible by RTM) + works with the latest Silverlight 1.1 Alpha. When the next preview of Silverlight releases this spring, we will release an update of the Astoria Silverlight client.

A number of people have asked me for a VB version of the CopyToDataTable<T> sample I wrote a few months back.   Unfortunately, between getting some skiing in (lots of snow in Washington this year!) and getting Astoria ready for Mix, I have not had very little free time.  Hence, if anyone else wants to take a crack at this - or knows of an existing port of the code to VB, let me know.