Skip to content

Correct sort order for tables with variable length key fields#4734

Merged
tillrohrmann merged 1 commit into
restatedev:mainfrom
tillrohrmann:fix-sort-order
May 18, 2026
Merged

Correct sort order for tables with variable length key fields#4734
tillrohrmann merged 1 commit into
restatedev:mainfrom
tillrohrmann:fix-sort-order

Conversation

@tillrohrmann
Copy link
Copy Markdown
Contributor

The problem is that variable length key fields are length encoded so that the length of the field decides its order and not the value. Hence, shorter values precede longer values which should violate DF sort order assumption.

@tillrohrmann tillrohrmann requested a review from igalshilman May 14, 2026 00:08
@tillrohrmann
Copy link
Copy Markdown
Contributor Author

I am probably misunderstanding the define_sort_order! macro @igalshilman. I thought that this macro would say which columns follow a natural sort order when reading the data from RocksDB. However, we seem to instantiate PhysicalSortExpr for those columns. In any case, I think that variable length fields have a problem as they are sorted by length and not lexicographical order in RocksDB.

@tillrohrmann tillrohrmann marked this pull request as ready for review May 18, 2026 09:27
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

The problem is that variable length key fields are length encoded so that
the length of the field decides its order and not the value. Hence, shorter
values precede longer values which should violate DF sort order assumption.
@tillrohrmann tillrohrmann merged commit c4b96f4 into restatedev:main May 18, 2026
18 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators May 18, 2026
@tillrohrmann tillrohrmann deleted the fix-sort-order branch May 18, 2026 09:36
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants