Implement testing for server package - #627
Draft
hpoeche wants to merge 35 commits into
Draft
Conversation
This is a draft to test unittest implementations for the server. Implement unittests using `werkzeug.test.Client` to test the `/shells` endpoint, as example. Each endpoint and method is tested for success and possible failures. The object store is reset prior to every test case. Tests are repeated for `application/json` and `application/xml` Content-Types. Therfore test are written against an abstract `FromatClient` that covers the details of (de-)serialization behind a simple API for requesting and parsing. Therefore the base class defining the test cases (`_ShellsEndpointTest`) is disabled for testing. Two subclasses are derived from this class, one for each format, that define the correct `FormatClient` and execute the tests.
…umbnail endpoint.
…ation thumbnail endpoint." This reverts commit 496190f, which held the content of eclipse-basyx#618. This was added for testing purpose only. Merge `develop` into this branch, after the PR was closed to obtain the same result.
For now the `/submodel-elements` paths are excluded
Added test class `TestPagination` to `test_base.py` that ensures pagination by following `cursor` value correctly assembles all items. Additionally, all endpoints, that should support pagination are checked if they do so.
To separate testing of the pagination logic from working endpoints, the shared function for creatin paginated responses is now tested directly. The base tests on paginated endpoints remain.
In the first version of the tests for the Discovery API, when data needed to be added to the DiscoveryStore, this was done through the `POST /lookup/shells/<aasId>` endpoint. To decouple endpoint tests from each other, the data insertion is now done directly via the DiscoveryStore.
…riptor writes" This reverts commit 46161d4. Creating issue for this to solve in later PR
Currently, the `DictDescriptorStore` used in the Registry when disabling persistent storage, throws an exception on `commit()` calls. To avoid failing pipeline and because persistent storage is more realistic end-user behavior, the integration tests now run with persistent storage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO
DiscoveryStoretest serde methodsfrom_file(), to_file()provider.py: functionload_directoryfor loadingDescriptorStorejsonization.py: Extendtest_registry.pycases to fullDescriptormodels (testing every branch of Encoder/Decoder)run_repository.py/run_registry.py: Testbuild_storage()respects environment variablesrun_discovery.py: Test loading file on run, saving on exitChanges
This PR adds unit and integration test to the previously untested server package. In detail the following test setup is used
Unit Testing
Unit tests are primarily written against the served API, employing
werkzeug.test.Clientto test thewerkzeugbased API interfaces.All three server interfaces (
repository,discovery,registry) are tested this way.Format specific testing
All implemented endpoints are tested with all supported
Content-Types and for all possible responses.Responses are not deserialized using the SDK adapter, because round-trip serialization-deserialization is already tested in the SDK. However, the correctness of responses are still verified by parsing them using standard libraries (
jsonandlxml.etree) and checking for specific expected data.To reduce repetition of test for both Content-Types
application/jsonandapplication/xml,test/interfaces/format_utils.pydefines a format-agnostic wrapperFormatClientofwerkzeug.test.Clientthat has subclassesJsonFormatClientandXmlFormatClienthandling serialization of request bodies as well as parsing and checking as described above.The test cases are written against this format-agnostic
FormatClient. Function decoratorswith_json_client/with_xml_clientare used to create separate test cases with the correct client at runtime.Coverage
In general, endpoints are tested for successful response and all error codes, defined in the specification. More specifically:
GETendpoints are tested forPOSTendpoints are tested for*ObjectStorePUTendpoints are tested for*ObjectStoreis altered correctlyDELETEendpoints are tested for*ObjectStoreAdditional Tests
While the test above test if the endpoints behavior is compliant to the specification, some additional tests directly test the code in
base.py:base.py, therefore we extensively test it separate for correct logic (pages build a partition of complete set) and parameter handling.JsonResponseandXmlResponseis tested separatelyHTTPApiDecoder), is already tested in the test cases above.Integration Testing
TODO: add description