The Axeda Platform has long had the ability to write custom logic to retrieve, manipulate and create data. In the current versions of the Platform, there are two classes of API, Version 1 (v1) and Version 2 (v2). The v1 APIs allow a developer to work with data on the Platform, but all of the APIs are subject to the maxQueryResults configuration property, which by default limits the number of results per query to 1000. For some subsets of data, this can be inadequate to process data. In comes the v2 API, which introduces pagination.
One of the first things a new user does when exploring the V2 API, is something like the following:
HistoricalDataItemValueCriteria criteria = new HistoricalDataItemValueCriteria()
criteria.assetId = '9701'
criteria.startDate = '2014-07-23T12:33:00Z'
criteria.endDate = '2014-07-23T12:44:00Z'
DataItemBridge dbridge = com.axeda.sdk.v2.dsl.Bridges.dataItemBridge
FindDataItemValueResult results = dbridge.findHistoricalValues(criteria)
And they get frustrated when they only get the same 100 rows of data. Repeat after me: V2 API invocations (find operations) are limited to batches of 100 results at a time!
But that's not the end of the story. With a small change, the query above can be tuned to iterate through all results that match the search criteria:
HistoricalDataItemValueCriteria criteria = new HistoricalDataItemValueCriteria()
criteria.assetId = '9701'
criteria.startDate = '2014-07-23T12:33:00Z'
criteria.endDate = '2014-07-23T12:44:00Z'
criteria.pageNumber = 1
criteria.pageSize = 100 // Default.
DataItemBridge dbridge = com.axeda.sdk.v2.dsl.Bridges.dataItemBridge
FindDataItemValueResult results = dbridge.findHistoricalValues(criteria)
tcount = 0
while ( (results = dbridge.findHistoricalValues(criteria)) != null && tcount < results .totalCount) {
results.dataItems.each { res ->
tcount++
}
criteria.pageNumber = criteria.pageNumber + 1
}
I currently recommend that people avoid using the count() or countDomainObjectByCriteria() functions if you're then going to call a find. Currently both the count*() and find functions compute total results, and doubles execution time of just those two calls. Total count is only computed when running the first find() operation, so the code pattern above is so far the most efficient way I've seen to run these operations on the platform.
So having covered how to do this in code (custom objects), let's turn our attention to the REST APIs - the other entry-point for using these capabilities. The REST API doesn't offer a count*() function, but the first find() invocation (if using XML) brings back totalCount as part of the result set. You can use this in your application to decide how many times to call the REST end-point to retrieve your data. So for the example above:
POST: https://customer-sandbox.axeda.com/services/v2/dataItem/findHistoricalValues
HEADERS:
Content-Type: application/xml
Accept: application/xml
BODY:
<?xml version="1.0" encoding="UTF-8"?>
<HistoricalDataItemValueCriteria xmlns="http://www.axeda.com/services/v2" pageSize="100" pageNumber="1">
<assetId>9701</assetId>
<StartDate>2014-07-23T12:33:00Z</StartDate>
<endDate>2014-07-23T12:35:02Z</endDate>
</HistoricalDataItemValueCriteria>
RESULTS:
<v2:FindAssetResult totalCount="1882" xmlns:v2="http://www.axeda.com/services/v2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<v2:criteria pageSize="100" pageNumber="1">
<v2:name>*</v2:name>
<v2:propertyNames/>
</v2:criteria>
<v2:assets>
</v2:assets>
</v2:FindAssetResult>
Or JSON:
POST: https://customer-sandbox.axeda.com/services/v2/dataItem/findHistoricalValues
HEADERS:
Content-Type: application/xml
Accept: application/xml
BODY:
{
"id": 9701,
"startDate": "2014-07-23T12:33:00Z",
"endDate": "2014-07-23T12:35:02Z",
"pageNumber": 1,
"pageSize": 2
}
And that's how you work around the maxQueryResults limitation of the v1 APIs. Some APIs do not currently have matching v2 Bridges (e.g. MobileLocation and DataItemAssociation), in which case the limitation will still apply. Creative use of the query Criteria will allow you to work around these limitations as we continue to improve the V2 API.
Regards,
-Chris
