e2e_part2 by AYAHASSAN287 · Pull Request #179 · logos-messaging/logos-delivery-interop-tests

AYAHASSAN287 · 2026-04-26T16:11:19Z

Summary

End-to-end test coverage for the wrapper send path, plus the supporting infrastructure (helpers, CI pipeline, store assertions). Adds 19 new scenarios across two files

Tests

tests/wrappers_tests/test_send_e2e_part1.py (TestSendBeforeRelay):

ID	Scenario
S17	Send before any relay peer joins; positive/negative paths
S19	Store peer comes online after propagation — Sent fires once reachable
S20	Relay/lightpush propagation succeeds, store query initially misses, retry republishes and validates
S21	Delivery retry window expires before any valid path recovers
S22	Non-ephemeral message with `reliabilityEnabled=False` — Propagated only, no Sent
S23	Non-ephemeral, reliability enabled, no store peer reachable — Propagated only, no Sent, no terminal error
S24	Ephemeral message with reachable store peer — Propagated only, no store validation
S26	Lightpush peer churn — selected peer disappears, alternate remains, message still propagates
S30	Concurrent sends on the same content topic during initial auto-subscribe — unique request IDs, no cross-association
S31	Concurrent sends across mixed topics during 3-phase peer churn (before / during / after restart)

tests/wrappers_tests/test_send_e2e_part2.py:

ID	Scenario
S02	Send without explicit subscribe — auto-subscribe on first send
S06	Core sender, relay only — propagation without store
S07	Core sender, relay + store — propagation with store validation
S08	Relay-to-lightpush fallback
S09	Relay-to-lightpush fallback with store validation
S10	Edge sender, lightpush only
S12	Isolated sender, no peers — error event
S14	Lightpush non-retryable error (oversized message)
S15	Lightpush retryable error then recovery

New helpers

src/node/wrapper_helpers.py — EventCollector (thread-safe async event sink), event-type predicates, wait_for_propagated/sent/error/connected, assert_event_invariants, get_node_multiaddr, create_message_bindings.
src/steps/store.py — check_sent_message_is_stored to assert hashes are present in store.
src/node/wrappers_manager.py — get_node_info_raw (returns the unparsed string).

Infra / wiring

New .github/workflows/pr_tests.yml — build (cached on submodule hash) → wrapper tests → smoke; full 18-shard suite triggered by full-test label.

Known issues

S19 __ failing in the post-propagation reliability/store phase: the message is propagated, but the sender cannot complete store validation before the retry window expires although Store peer joins and is detected:``
storeClientCount=1 issue
S31 — json_serialization/writer.nim → w.inObject() AssertionDefect and nwaku SIGSEGV during shutdown/churn. Note: flaky not show up always
S20 — when the store peer joins after the initial miss, the retry loop fails to republish: sender repeatedly logs NoPeersToPublish and eventually emits message_error: Unable to send within retry time window.
note: the main point is that it's not possible to force the miss round
S08__ the test fails if lightpushnode not used issue

* - Add S06 relay-only test case for testing message propagation without a store. - Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage). - Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`. - Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method. - Update `.gitignore` to exclude `store.sqlite3*`. * Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness. * Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

… helper (#178) - Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop. - Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`. - Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios. Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests * Integrate `assert_event_invariants` into edge and store tests * Remove redundant comments from `test_send_e2e.py` --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

fbarbu15

LGTM, thanks but can you please add a full run test report to the PR description?

* Add tests for auto-subscribe on first send and isolated sender with no peers * Add PR CI workflow with tiered test strategy - pr_tests.yml: build job with cache, wrapper-tests, smoke-tests, and label-triggered full-suite - test_common.yml: add deploy_allure/send_discord inputs so PR runs skip reporting side effects - Add docker_required marker to S19 (needs Docker, excluded from wrapper-only CI job) - Register docker_required marker in pytest.ini * Document PR CI test workflows in README * Refine PR CI test strategy: - Exclude `docker_required` tests from smoke set in `pr_tests.yml`. - Add `wait_for_connected` helper for connection state checks. - Update S19 test to dynamically create and clean up the store node setup. - General simplifications and improved test stability. * Add `wait_for_connected` assertion to ensure sender connection state before propagation test * Refine tests and CI workflows: - Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`. - Adjust timeout assertion for better clarity and accuracy. - Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests. - Change `test_common.yml` default Discord reporting to `false`. * Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

at0m1x19

A few comments — mostly suggestions, nothing blocking the overall direction. Thanks for the work on this!

at0m1x19

Follow-up batch — a question on the xfail markers, a validated note on docker_required, and a couple of unused imports.

at0m1x19

Final batch of part1 comments

AYAHASSAN287 · 2026-05-07T08:55:23Z

LGTM, thanks but can you please add a full run test report to the PR description?

latest job
here is already 2 open bugs related to the open issues

…htpush peer discovery.

…to broken lightpush peer discovery.

AYAHASSAN287 · 2026-05-11T11:25:06Z

LGTM, thanks but can you please add a full run test report to the PR description?

Latest run , all skipped tests failing for issues opened

Ivansete-status

Sorry for late review. Just adding a few comments. Thanks!

Ivansete-status · 2026-05-12T07:52:02Z

+
+            logger.debug(f"messages length is {len(self.store_response.messages)}")
+            assert self.store_response.messages, f"Peer {node.image} couldn't find any messages. " f"Actual response: {self.store_response.resp_json}"
+            assert len(self.store_response.messages) >= len(expected_hashes), (


Isn't it better to assert strict equal?

The real correctness check is the loop (assert expected_hash in actual_hashes)
Store state isn't isolated ,the store node can hold messages from prior store requests .

Ivansete-status · 2026-05-12T07:58:32Z

+                "tcpPort": free_port(),
+                "discv5UdpPort": free_port(),
+                "restPort": free_port(),


After having merged the following, it should be enough to pass 0 to these to perform auto-port internally. Maybe another test to consider in the future: logos-messaging/logos-delivery#3828

I will add it in recent tests , add specific test for it

Ivansete-status · 2026-05-12T08:02:43Z

+                "relay": True,
+                "store": False,
+                "lightpush": False,
+                "filter": False,


Is interesting to only configure nodes by only touching the mode attribute, i.e., only setting either mode: "Edge" or mode: "Core". Even though nowadays is supported to manually set protocols, we aim for devs to not tweak that directly, in the future.
I mean, this is actually a flaw in the API, that should not accept those attributes, and we might fix that in the future.

I considered this in the recent PR

AYAHASSAN287 and others added 13 commits April 14, 2026 19:43

add test s17

f3f4cf8

Add temp changes

2580749

Merge remote-tracking branch 'origin/master' into temp_bindings_branch

93f2d2a

Add s17 positive / negative scenarios

3d7a92c

add S19

73884ee

Modify S19 test

34cfb80

Adding S21

e27d64a

Fix review comments

060d50a

Merge branch 'master' into temp_bindings_branch

91a9fc5

Adding S22/S23

572b50c

Adding S24

9e11ccb

Add S26

fb957dd

AYAHASSAN287 changed the title ~~E2 e part2~~ e2e_part2 Apr 26, 2026

AYAHASSAN287 and others added 5 commits April 27, 2026 10:42

Add S30

f6a558b

Add S31

8c7979c

Fix some tests

16bef03

AYAHASSAN287 requested review from at0m1x19 and fbarbu15 April 30, 2026 11:34

fbarbu15 reviewed Apr 30, 2026

View reviewed changes

at0m1x19 and others added 8 commits April 30, 2026 16:15

Add relay-to-lightpush fallback integration tests (S08/S09) (#180)

f219c80

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

Ignore S19

5d1614f

fix s26

9191be1

Ignore s20 / s31 for errors

882b186

Change image name

4aa64c7

fix xfail syntax error

ee6816c

rename test file

507ddfa

Modify S20

8e639c1

at0m1x19 reviewed May 4, 2026

View reviewed changes

Comment thread .github/workflows/pr_tests.yml Outdated

Comment thread tests/wrappers_tests/test_send_e2e_part2.py

Comment thread tests/wrappers_tests/test_send_e2e_part2.py Outdated

Comment thread src/node/wrapper_helpers.py

fix portsshift option

55f9612

at0m1x19 reviewed May 4, 2026

View reviewed changes

Comment thread tests/wrappers_tests/test_send_e2e_part1.py Outdated

Comment thread tests/wrappers_tests/test_send_e2e_part1.py

Comment thread tests/wrappers_tests/test_send_e2e_part1.py

Comment thread tests/wrappers_tests/test_send_e2e_part2.py Outdated

at0m1x19 reviewed May 4, 2026

View reviewed changes

AYAHASSAN287 mentioned this pull request May 4, 2026

bug: Reliability layer fails to republish after store peer joins mesh — S19 times out waiting for MessageSentEvent logos-messaging/logos-delivery#3848

Open

AYAHASSAN287 added 2 commits May 5, 2026 10:05

remove the /true from yml to allow errors to exist

c6b5a02

Modify the yml to continue on error

da655b4

AYAHASSAN287 added 10 commits May 7, 2026 11:21

First set of review comments

c34148b

adding xfail mark for failed tests

2de9b03

address review comments about xfail

bf17318

cleanup unused lines

92aeeaa

event collector fix

30c426f

Address review comment about delay constant

505d24f

fix the timeout review comment

5d0c283

Add assert_event_invariants

ccba9ce

enhance comment on S26 test

892ff96

mark the waku tests as docker_required

4ad51bf

AYAHASSAN287 requested a review from at0m1x19 May 7, 2026 13:12

Egor Rachkovskii added 2 commits May 11, 2026 09:41

Mark test_s10_edge_lightpush_propagation as xfail due to broken lig…

841e7d4

…htpush peer discovery.

Mark test_s15_lightpush_retryable_error_then_recovery as xfail due …

30de188

…to broken lightpush peer discovery.

at0m1x19 approved these changes May 11, 2026

View reviewed changes

Merge remote-tracking branch 'origin/master' into E2E_part2

0306e6d

AYAHASSAN287 requested a review from fbarbu15 May 11, 2026 11:23

fbarbu15 approved these changes May 11, 2026

View reviewed changes

AYAHASSAN287 merged commit 11197db into master May 11, 2026
4 of 5 checks passed

AYAHASSAN287 deleted the E2E_part2 branch May 11, 2026 13:53

Ivansete-status reviewed May 12, 2026

View reviewed changes

Conversation

AYAHASSAN287 commented Apr 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

New helpers

Infra / wiring

Known issues

Uh oh!

fbarbu15 left a comment

Choose a reason for hiding this comment

Uh oh!

at0m1x19 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

at0m1x19 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

at0m1x19 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AYAHASSAN287 commented May 7, 2026

Uh oh!

AYAHASSAN287 commented May 11, 2026

Uh oh!

Uh oh!

Ivansete-status left a comment

Choose a reason for hiding this comment

Uh oh!

Ivansete-status May 12, 2026

Choose a reason for hiding this comment

Uh oh!

AYAHASSAN287 May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Ivansete-status May 12, 2026

Choose a reason for hiding this comment

Uh oh!

AYAHASSAN287 May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Ivansete-status May 12, 2026

Choose a reason for hiding this comment

Uh oh!

AYAHASSAN287 May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AYAHASSAN287 commented Apr 26, 2026 •

edited

Loading

at0m1x19 left a comment •

edited

Loading