Daniel Doubrovkine bio photo

Daniel Doubrovkine

aka dB., @ShopifyEng, @OpenSearchProj, ex-@awscloud, former CTO @artsy, +@vestris, NYC

Email Twitter LinkedIn Github Strava
Suggest a Topic
RSS
Creative Commons License

Fixing Bugs in the OpenSearch API Documentation using API Spec Tests
opensearch

I’ve previously written about the OpenSearch OpenAPI Specification.

One of the neat features of the API spec repo is a set of declarative YAML integration tests. This turned out to be really useful in fixing bugs in the OpenSearch API reference documentation that is published to opensearch.org.

Here’s how I went about it.

To author tests, I started with reading the API documentation. For example, I read the docs for /_refresh, and added tests for that API in opensearch-api-specification#374.

The basic example that refreshes all indices was easy.

$schema: ../../json_schemas/test_story.schema.yaml  description: Test _refresh.  chapters:   - synopsis: Refresh.     path: /_refresh     method: POST     response:       status: 200 
$ npm run test:spec--insecure -- --tests tests/indices/refresh.yaml  > [email protected] test:spec--insecure > ts-node tools/src/tester/test.ts --opensearch-insecure --tests tests/indices/refresh.yaml  OpenSearch 2.15.0  PASSED  refresh.yaml (.../tests/indices/refresh.yaml)

Then, I tried using the query parameters.

- synopsis: Refresh an index.     path: /{index}/_refresh     method: POST     parameters:       index: movies       ignore_unavailable: true       allow_no_indices: false       expand_wildcard: all # incorrect

This failed.

$ npm run test:spec--insecure -- --tests tests/indices/refresh.yaml  > [email protected] test:spec--insecure > ts-node tools/src/tester/test.ts --opensearch-insecure --tests tests/indices/refresh.yaml  OpenSearch 2.15.0  ERROR   refresh.yaml (.../tests/indices/refresh.yaml)     ERROR   CHAPTERS         PASSED  Refresh.             PASSED  REQUEST BODY             PASSED  RESPONSE STATUS             PASSED  RESPONSE PAYLOAD BODY             PASSED  RESPONSE PAYLOAD SCHEMA         ERROR   Refresh an index.             FAILED  PARAMETERS                 PASSED  ignore_unavailable                 PASSED  allow_no_indices                 FAILED  expand_wildcard (Schema for "expand_wildcard" parameter not found.)             PASSED  REQUEST BODY             ERROR   RESPONSE STATUS (Expected status 200, ...)             SKIPPED RESPONSE PAYLOAD BODY             SKIPPED RESPONSE PAYLOAD SCHEMA

The failure is with the expand_wildcard parameter, which is misspelled and should be expand_wildcards.

- synopsis: Refresh an index.     path: /{index}/_refresh     method: POST     parameters:       index: movies       ignore_unavailable: true       allow_no_indices: false       expand_wildcards: all     response:       status: 200

The corrected test passes.

$ npm run test:spec--insecure -- --tests tests/indices/refresh.yaml --verbose  > [email protected] test:spec--insecure > ts-node tools/src/tester/test.ts --opensearch-insecure --tests tests/indices/refresh.yaml  OpenSearch 2.15.0  PASSED  refresh.yaml (.../tests/indices/refresh.yaml)     PASSED  CHAPTERS         PASSED  Refresh.             PASSED  REQUEST BODY             PASSED  RESPONSE STATUS             PASSED  RESPONSE PAYLOAD BODY             PASSED  RESPONSE PAYLOAD SCHEMA         PASSED  Refresh an index.             PASSED  PARAMETERS                 PASSED  ignore_unavailable                 PASSED  allow_no_indices                 PASSED  expand_wildcards             PASSED  REQUEST BODY             PASSED  RESPONSE STATUS             PASSED  RESPONSE PAYLOAD BODY             PASSED  RESPONSE PAYLOAD SCHEMA

All we have left is to fix the documentation in documentation-website#7620. I also fixed a bug in PUT mapping in documentation-website#7652 and corrected the type of include_defaults in GET /_settings in documentation-website#7657.

Finally, we’d like to generate a lot of the documentation from spec in documentation-website#7700.