/ 
WebAPI-DBIC-0.004002
River stage one • 1 direct dependent • 1 total dependent
/ data_service_api

AUTHENTICATION

Currently we're using HTTP Basic authentication and using the supplied credentials as the credentials for database access. So whatever username and password are provided with the request will be used to perform the request on the database.

Clearly https is needed if auth is required and the interface is public!

Current Basic auth leaves db password in browsers auth cache or keychain.

Digest auth http://en.wikipedia.org/wiki/Digest_access_authentication
http://search.cpan.org/perldoc?Plack%3A%3AMiddleware%3A%3AAuth%3A%3ADigest

Also http://stackoverflow.com/questions/319530/restful-authentication

IMPLEMENTATION NOTES

We use the existing DBIx::Class classes directly as the entity definitions exposed by the data service. This is reasonable given that it's an internal service so there's little value in abstracting it further.

The pattern of set, set/id, set/id/relation, set/id/relation/id can continue to any depth but we're using a flat namespace for now. If we used a deeper path it would represent 'is identified by' (or 'belongs to') relationships. This complicates the caching/invalidation though, so we'll keep it flat for now.

XXX Allow overriding the method via a url param, e.g. for testing: GET /dogs?method=delete TBD

XXX Allow overriding the return status via a url param, e.g. for testing: GET /dogs?suppress_response_codes=true would return 200 status, but the content would still be the error details.

ETag etc [LATER]

For datetime use UTC and ISO8601 - automation via TBD

Review the Web Linking spec (RFC5899) and HAL for use of rel links, eg first/prev/next/last. [LATER]

Formalize the error response contents.

Validation

Validation in the data service requires defining an approach to parameter validation (eg adopting a module like Params::Validate plus Moose attribute validations) and defining an approach to throwing and handling exceptions.

Specifically it should provide sufficient information to the client, via the JSON response, to enable the client to update the form to indicate *which* field(s) are associated with the error.

Similarly exceptions raised due to database constraint errors should also generate client-useful exceptions with field information if possible. Note that this may require some mapping of database field names to json entity type field names.

CACHING

Implementing caching is easy. Implementing efficient caching (where each resource is only cached one - a canonical copy) and cache invalidation (eg a trigger on a table can invalidate the cached copy of affected rows) is, er, non-trivial.

ESI

Edge Side Include is the "secret sauce" that enables caching (and cache invalidation) to work nicely with prefetch and HAL.

Consider a simple request like /foo/42. The response looks like:

{
    id: 42,
    ...foo fields...
}

A database trigger on the foo table could be used to invalidate the cache for a particular /foo/:id record when that record is updated. So far so good.

Now consider a request with prefetch: /foo/42&prefetch=bar where the response looks like:

{
    id: 42,
    ...other foo fields...
    _embedded: {
        bar: { id: 97, ...other bar fields...}
    }
}

Now there's no simple way to invalidate that cached response when the corresponding record in the bar table is updated.

This is where ESI comes in. The response from the API would look like this:

{
    id: 42,
    ...other foo fields...
    _embedded: {
        bar: <esi:include src="/bar/97">
    }
}

the ESI processor (eg varnish) caches that unprocessed response and then processes the ESI requests embedded in it. So it makes a separate request for "/bar/97" (which may well be resolved from its own cache) and builds the response to send to the client.

The same database triger mechanism on the bar table will invalidate the cached /bar/97 response when the corresponding record in the bar table is updated.

With ESI, this invalidation affects all cached responses.

http://odino.org/some-common-questions-about-edge-side-includes/
http://stackoverflow.com/questions/11781576/most-secure-javascript-json-inline-technique

Also look into "Surrogate-Capability & Surrogate- Control headers for ESI based block caching"

Varnish

The varnish cache (see http://varnish-cache.org) supports basic ESI and also enables alternative approaches that might be useful:

* X-depends-on - e.g. http://www.smashingmagazine.com/2014/04/23/cache-invalidation-strategies-with-varnish-cache/

* https://www.varnish-cache.org/utilities?field_utility_category_tid=16

* http://www.hward.com/varnish-cache-invalidation-with-fastly-surrogate-keys

RESEARCH

REST Core concepts and specifications:

http://en.wikipedia.org/wiki/Representational_State_Transfer
http://www.w3.org/Protocols/rfc2616/rfc2616.html (HTTP Spec)
https://github.com/basho/webmachine/wiki/Diagram

Best practice (hint: there's isn't one, just lots of suggestions):

http://www.slideshare.net/Wombert/designing-http-interfaces-and-restful-web-services-sfliveparis2012-20120608
http://www.infoq.com/articles/webber-rest-workflow
https://restful-api-design.readthedocs.org/en/latest/
http://www.stormpath.com/blog/designing-rest-json-apis
http://www.slideshare.net/guilhermecaelum/rest-in-practice (XML)
http://www.slideshare.net/apigee/restful-api-design-second-edition (107 slides)
http://www.foxycart.com/blog/the-hypermedia-debate#.UT8PSKVYXdk

PUT vs POST

http://jcalcote.wordpress.com/2008/10/16/put-or-post-the-rest-of-the-story/
http://benramsey.com/blog/2009/11/post-vs-put/
http://techno-weenie.net/2011/4/28/my-put-requests-bug-me/

Example APIs

http://developer.github.com/v3/
http://bitworking.org/projects/atom/rfc5023.html

API Design

http://www4.in.tum.de/~blanchet/api-design.pdf

Linking

http://amundsen.com/media-types/linkrelations/
http://www.iana.org/assignments/link-relations/link-relations.xml
http://tools.ietf.org/html/rfc5988
http://www.mnot.net/blog/2011/11/25/linking_in_json (see also the comments)

HAL - Hypertext Application Language

http://blog.stateless.co/post/13296666138/json-linking-with-hal
http://stateless.co/hal_specification.html
http://tools.ietf.org/html/draft-kelly-json-hal
http://haltalk.herokuapp.com/explorer/browser.html
http://www.quora.com/REST-software-architectural-style/JSON-+-Hypermedia-Using-HAL-in-Production

URI Template

http://tools.ietf.org/html/rfc6570
https://metacpan.org/module/URI::Template

CURIE Syntax - Compact URIs

http://www.w3.org/TR/curie/

Partial reponses

http://blog.apigee.com/detail/restful_api_design_can_your_api_give_developers_just_the_information
https://developers.google.com/+/api/#partial-response

Other references:

http://www.programmableweb.com
http://www.programmableweb.com/apis/directory/1?protocol=REST&format=JSON
http://www.slideshare.net/jmusser/j-musser-apishotnotgluecon2012
http://nocarrier.co.uk/2012/09/hypermedia-types-and-connection-negotiation/

Restful Objects:

http://en.wikipedia.org/wiki/Restful_Objects
http://www.infoq.com/articles/Intro_Restful_Objects
Demo: http://simple-dusk-6870.herokuapp.com/arow-ronet.html#
http://skillsmatter.com/podcast/design-architecture/restful-objects (video)

Assorted Proposed Standards

http://json-ld.org

https://github.com/kevinswiber/siren

http://librelist.com/browser//hypermedia/2012/5/2/notes-on-hal-and-collection+json/

Error Formats

https://github.com/blongden/vnd.error
http://tools.ietf.org/html/draft-nottingham-http-problem-02