In this post I’m doing a deep dive of AWS DynamoDB database and using the available APIs with .Net Core. For more general information on DynamoDB, refer to this post:
http://solidfish.com/overview-of-aws-dynamodb/
The code shown here is from another post where I created a complete serverless web application in AWS using DynamoDB. Below is a link to that post as well as the github project site.
http://solidfish.com/serverless-app-using-aws-api-dynamodb-lambda-s3-visual-studio-dot-net/
https://github….tbd
To work with the DynamoDB in .Net we need the AWS SDK installed on Visual Studio. The SDK provides a few different ways to communicate with DynamoDB. The diagram below shows an overview of these approaches. The actual communication between the SDK and DynamoDB database is done over HTTPS.
Interfaces
The application works with DynamoDB through the 3 different interfaces shown in the diagram. Each interface has different pros/cons to be used for different use cases.
- Low-Level Interface (Amazon.DynamoDBv2)
- All AWS SDKs provide low-level interface and this is the interface for DynamoDB. At this level you may need to explicitly call out data types. This requires some more coding at the application layer and may cause complexities. However, the benefit is that at this level we have full access to all the APIs. A list of these APIs can be found here:
- https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations_Amazon_DynamoDB.html
- Document Model (Amazon.DynamoDBv2.DocumentModel)
- The document model provides a simpler interface using a Table object for the table and a Document object for the rows. With these objects it becomes easier to code at the application layer, however there can be some limitations using these objects. The benefit is that by interacting with the Document and Table object, the data conversions are done for you.
- Object Persistence Model (Amazon.DynamoDBv2.DataModel)
- With the Object Persistence model we use the DynamoDBContext to interact with DynamoDB. This allows us to use .Net models to be stored on the database. The models must match the target tables hash/range keys but other fields are optional. The API will automatically convert the other data types. This a great option when trying to map .Net objects (models) against the DynamoDB. However, with the Context model there are some limitations in functionality – for example it cannot make conditional Put calls.
Low-Level API
At the lowest level of the DynamoDB SDK we see the Low-Level API which contains the actual HTTPS request/response handling. This should not be confused with the Low-Level Interface which is implemented with the AmazonDynamoDB client class. The Low-Level API is where the actual communication takes place using JSON format and the API predefined data types. Some of these data type descriptors are listed below. For binary data, it is encoded in Base64 format (RFC 4648).
- S – string
- N – number
- B – binary
- BOOL – boolean
- NULL
- M – map
- L – list
- SS – string set
- NS – number set
- BS – binary set
As the low level API is using HTTPS, errors are usually responses with HTTP 400 or 500 codes. For complete list of error codes that can be return, refer to this article:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html
Implementing the Interfaces
Refer to the Serverless AWS app project post for a working example that uses the different interfaces in .Net. That post is here:
http://solidfish.com/serverless-app-using-aws-api-dynamodb-lambda-s3-visual-studio-dot-net/
In the Repository.cs file we have implemented the 3 different interface approaches for creating a record and then reading it. The sample code shows this below. Note that for the Low Level interface the data conversion had to be done whereas on the other two model based approaches the data conversion was ignored.
public async Task GetWidgetUsingLowLevel()
{
var widget = new Widget();
var putRequest = new PutItemRequest
{
TableName = "Widgets",
Item = new Dictionary<string, AttributeValue>
{
{ "Id", new AttributeValue { S = widget.Id } },
{ "Name", new AttributeValue { S = widget.Name } },
{ "Description", new AttributeValue { S = widget.Description } },
{ "Created", new AttributeValue { S = widget.Created.ToString() } },
{ "Price", new AttributeValue { S = widget.Price.ToString() } },
}
};
await _client.PutItemAsync(putRequest);
var getRequest = new GetItemRequest
{
TableName = "Widgets",
Key = new Dictionary<string, AttributeValue>
{
{ "Id", new AttributeValue { S = widget.Id } }
}
};
var getItem = await _client.GetItemAsync(getRequest);
var result = getItem.Item;
return result["Name"].S;
}
public async Task GetWidgetUsingDocumentModel()
{
var widget = new Widget();
var widgetTable = Table.LoadTable(_client, "Widgets");
var widgetDocument = new Document();
widgetDocument["Id"] = widget.Id;
widgetDocument["Name"] = widget.Name;
widgetDocument["Description"] = widget.Description;
widgetDocument["Created"] = widget.Created;
widgetDocument["Price"] = widget.Price;
await widgetTable.PutItemAsync(widgetDocument);
var resultDocument = await widgetTable.GetItemAsync(widget.Id);
return resultDocument["Name"].AsString();
}
public async Task GetWidgetUsingObjectPersistence()
{
var widget = new Widget();
await _context.SaveAsync(widget);
var result = await _context.LoadAsync(widget.Id);
return result.Name;
}
In the same Repository.cs file we also have some sample code that dives deeper into the Object Persistence approach. First we define the table using the Widget class. That is shown below:
[DynamoDBTable("Widgets")]
public class Widget
{
public Widget()
{
var id = "WDGT" + DateTime.Now.Ticks;
this.Id = id;
this.Name = $"Widget {id}";
this.Description = $"This is a widget {id}";
this.Created = DateTime.Now;
this.Price = (decimal) DateTime.Now.Second + DateTime.Now.Millisecond;
}
[DynamoDBHashKey]
public string Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public DateTime Created { get; set; }
public decimal Price { get; set; }
[DynamoDBProperty("Types")]
public List Types { get; set; }
public List Keywords { get; set; }
public override string ToString()
{
return string.Format(@"{0} : {1} - {2}", Id, Name, Description);
}
}
To interact with the DynamoDB table we use the DynamoDBContext class which is declared in the Repository class. This context model lets use interact with the tables including CRUD operations as well as running Queries and Scans. Some of the methods it provides are:
- CreateMultiTableBatchGet
- CreateMultiTableBatchWrite
- CreateBatchGet
- CreateBatchWrite
- Delete
- Dispose
- ExecuteBatchGet
- ExecuteBatchWrite
- FromDocument
- FromQuery
- FromScan
- GetTargetTable
- Load
- Query
- Save
- Scan
- ToDocument
Optimistic Locking
To avoid updates to stale record copies the DynamoDBTable attribute allows optimistic locking by defining a version number field. This is done with the DynamoDBVersion attribute. With this version number we can determine older copies. The AWS SDK will perform the lock (its more of a programmatic lock from the .Net code instead of directly in DynamoDB, which does not have locking features).
In the Repository.cs class we demonstrate the following features offered through the Object Persistence model.
- Create, Update and Deleting
- Mapping Arbitrary Data
- Batch Operations
- Scans
- Queries
Please refer to the github project site to see the complete file and these examples. The following post explains that github project and how it works.
http://solidfish.com/serverless-app-using-aws-api-dynamodb-lambda-s3-visual-studio-dot-net/
References
AWS SDK .NET Reference Document
https://docs.aws.amazon.com/sdkfornet/v3/apidocs/
DynamoDB API Blog
https://aws.amazon.com/blogs/developer/dynamodb-apis/
Programming with DynamoDB
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.html
Working with DynamoDB Object Persistence Model in .Net
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DotNetSDKHighLevel.html