chore: Add E2E test for path server demos

[arjo129]

Currently our tests do not exercise the path_server_demo. This has lead
to regressions when testing (particularly with DDS) as our integration tests
may pass but the first demo which people try is broken. In this test we
check the REST endpoints of the demo correctly integrate with the rest
of the system.

GenAI Use

We follow OSRA's policy on GenAI tools

• I used a GenAI tool in this PR.
• I did not use GenAI

Generated-by:

[mxgrey]

@SamuelFoo please review

[SamuelFoo]

The changes look good but maybe we need to specify rmf_path_server_demo for the CI's colcon test --packages-select?
Also, correct me if I'm wrong, but should the base branch be main instead?

(I don't seem to have the option to submit a review, so I’m leaving this as a comment.)

[arjo129]

The changes look good but maybe we need to specify rmf_path_server_demo for the CI's colcon test --packages-select? Also, correct me if I'm wrong, but should the base branch be main instead?

(I don't seem to have the option to submit a review, so I’m leaving this as a comment.)

Thanks for catching that.

[cardboardcode]

Progress Update 💬 cc @arjo129

Reviewed. The current proposed test_spawner_e2e.py test end-to-end by playing out the following scenario:

1. Spawns 2 robots, namely robot_1 and robot_2.
2. Sets destinations for both robots.
3. Triggers scenario to make both robots move.
4. Evaluates the following:
   • Has robots spawned successfully?
   • Has destination been set successfully?
   • Has scenario been triggered successfully?
   • Has robots reached within 0.3 meters of their respective destinations.

Steps To Run Local CI Workflow 📘

Follow the steps below to run the CI locally.

cd $HOME

git clone https://github.com/open-rmf/next_gen_prototype --branch arjoc/feat/web_demo_regression_tests --single-branch --depth 1 && cd next_gen_prototype

act -v -j ci-check --reuse --pull=false

Verify ✅

By running the commands, you should get a similar output like what is shown below:

...
| ============================== 5 passed in 14.59s ==============================
Finished <<< rmf_reservation_tests [15.3s]                 s]
| 
| Summary: 3 packages finished [1min 41s]
|   1 package had stderr output: rmf_path_server_test
| build/rmf_path_server_demo/pytest.xml: 1 test, 0 errors, 0 failures, 0 skipped
| build/rmf_path_server_test/pytest.xml: 5 tests, 0 errors, 0 failures, 0 skipped
| build/rmf_reservation_tests/pytest.xml: 5 tests, 0 errors, 0 failures, 0 skipped
| 
| Summary: 11 tests, 0 errors, 0 failures, 0 skipped
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2]   ✅  Success - Main Run Automated Verification Tests [2m50.345929732s]
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] skipping post step for 'Checkout Prototype Repository': no action model available
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] ⭐ Run Complete job
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] Cleaning up container for job Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] Loading revision from git directory
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] Found revision: 5e1ca9b531ac441b5370e272d6ffd44dd4b5b694
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] HEAD points to '5e1ca9b531ac441b5370e272d6ffd44dd4b5b694'
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] using github ref: refs/heads/arjoc/feat/web_demo_regression_tests
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] [DEBUG] Found revision: 5e1ca9b531ac441b5370e272d6ffd44dd4b5b694
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2]   ✅  Success - Complete job
[Next-Gen Prototype CI/Colcon Overlay Build & Test Suite (rmw_cyclonedds_cpp)-2] 🏁  Job succeeded

Follow-Up Action 💬

@arjo129
Think the e2e tests are clear and evaluates what is needed for the web demo. ✅

Similar to another similar python-based ROS 2 package, rmf_path_server_test , would suggest to add the following files under directory /path_server/rmf_path_server_demo/test just to make sure the static analysis is coherent for as many python packages in this repository as possible.

1. test_copyright.py Reference Link
2. test_flake8.py Reference Link
3. test_pep257.py Reference Link

After adding the above files, run the steps above to ensure the CI still passes locally.

Once CI passing again, should be good to merge.

[cardboardcode]

Mostly looks good. ✅

Please add the files and ensure CI passes as requested in Follow-Up Action section of previous comment.

[cardboardcode]

LGTM. ✅ Ready to merge.

However, one thing of note, when running the CI locally, the flakiness of the following unit tests, given their use of arbitrary timeouts, seems significant to cause the CI workflow, ci_check, to fail sometimes, given a lower spec device, and therefore can be further reduced:

1. test_spawner_two_robot_scenario
2. test_follow_scenario
3. test_scenario
4. test_plan_generation_and_publication
5. test_single_reservation
6. test_robot_sim_behavior

Will open a new PR to address this in the very near future, as this issue is not confined to only rmf_path_server_demo which is the main focus of this thread.

[arjo129]

Can you tell me which rmw you're seeing flakiness on. GitHub runners are pretty low specced.