Skip to content

[CALCITE-7731] Bound plain-notation expansion of DECIMAL literals to prevent parse-time OutOfMemoryError - #5204

Open
rubenada wants to merge 3 commits into
apache:mainfrom
rubenada:CALCITE-7731
Open

[CALCITE-7731] Bound plain-notation expansion of DECIMAL literals to prevent parse-time OutOfMemoryError#5204
rubenada wants to merge 3 commits into
apache:mainfrom
rubenada:CALCITE-7731

Conversation

@rubenada

Copy link
Copy Markdown
Contributor

Jira Link

CALCITE-7731

Changes Proposed

BigDecimal accepts any int exponent, so a DECIMAL literal such as DECIMAL '1E2147483647' (~12 characters) parses to a BigDecimal whose plain-notation form would be one character per digit: a multi-gigabyte allocation (potentially an OOM error).

Three places call BigDecimal.toPlainString() on a value derived from user-supplied input and would attempt that allocation:

SqlParserUtil.parseDecimalLiteral
SqlNumericLiteral.toValue
RexBuilder.makeLiteral

It is required to add a check in there to prevent an OOM error.

@mihaibudiu mihaibudiu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easy to tell by eye whether all the places where the conversion happens have been instrumented.

I wonder whether the exception message size is a concern. Is the problem only when the original literal is small but the internal representation is large?

@Override public String toValue() {
final BigDecimal bd = getValueNonNull();
if (exact) {
if (!SqlUtil.isBoundedDecimal(bd)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this error message be very large too?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, well spotted. I have changed it into bd.toString() , which should be the less OOM-risky alternative.

@rubenada

Copy link
Copy Markdown
Contributor Author

Not easy to tell by eye whether all the places where the conversion happens have been instrumented.

I wonder whether the exception message size is a concern. Is the problem only when the original literal is small but the internal representation is large?

The problem is mostly when toPlainString method is used, whose OOM risk can be quite high.
I have searched manually the callers of this method, and I think the PR covers all places susceptible to have a potential OOM risky input value from the user. There was another occurrence in SqlImplementor, I've just included the check in there just in case.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants