Commit Graph

31238 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
9e51908b02 Merge pull request #7551 from MathiasVP/fix-join-orders-in-unsigned-difference-expr-query
C++: Fix join orders in `cpp/unsigned-difference-expression-compared-zero`
codeql-cli/v2.7.5
2022-01-12 08:29:03 +00:00
Mathias Vorreiter Pedersen
2a02ce137a C++: Fix join orders in 'exprIsSubLeftOrLess'.
Before:

Tuple counts for UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff/2@i3#a5071w3a after 24s:
  304220    ~2%      {2} r1 = JOIN UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff#prev_delta WITH Expr::BinaryOperation#class#f#join_rhs ON FIRST 1 OUTPUT Lhs.1, Rhs.0 'sub'

  190061335 ~24%     {2} r2 = JOIN r1 WITH DataFlowUtil::localFlowStep#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  3956      ~0%      {2} r3 = JOIN r1 WITH DataFlowUtil::localFlowStep#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  407983    ~1%      {2} r4 = JOIN Expr::BinaryOperation#class#f#join_rhs WITH UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff#prev ON FIRST 1 OUTPUT Rhs.1 'n', Lhs.0 'sub'
  380823    ~0%      {2} r5 = JOIN r4 WITH DataFlowUtil::TExprNode#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1
  0         ~0%      {2} r6 = JOIN r5 WITH UnsignedDifferenceExpressionComparedZero::isGuarded#fff#prev_delta ON FIRST 2 OUTPUT Rhs.2, Lhs.0 'sub'
  0         ~0%      {2} r7 = JOIN r6 WITH DataFlowUtil::TExprNode#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  3956      ~0%      {2} r8 = r3 UNION r7
  190065291 ~24%     {2} r9 = r2 UNION r8
  ...

After:

Tuple counts for UnsignedDifferenceExpressionComparedZero::interestingSubExpr#f/1@654e29g3 after 228ms:
  370 ~2%     {2} r1 = ComparisonOperation::RelationalOperation::getGreaterOperand_dispred#fb AND NOT Exclusions::isFromMacroDefinition#b(Lhs.1 'sub')
  370 ~0%     {2} r2 = SCAN r1 OUTPUT In.1 'sub', In.0
  370 ~3%     {3} r3 = JOIN r2 WITH Expr::Expr::getFullyConverted_dispred#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0 'sub'
  210 ~1%     {2} r4 = JOIN r3 WITH SimpleRangeAnalysis::SimpleRangeAnalysisCached::exprMightOverflowNegatively#f ON FIRST 1 OUTPUT Lhs.2 'sub', Lhs.1
  210 ~0%     {3} r5 = JOIN r4 WITH Expr::Expr::getFullyConverted_dispred#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'sub', Rhs.1
  210 ~1%     {3} r6 = JOIN r5 WITH ComparisonOperation::RelationalOperation::getLesserOperand_dispred#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'sub', Lhs.2
  59  ~2%     {4} r7 = JOIN r6 WITH Expr::Expr::getValue_dispred#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Lhs.2, Rhs.1, toInt(Rhs.1)
  17  ~0%     {4} r8 = SELECT r7 ON In.3 = 0
  17  ~0%     {2} r9 = SCAN r8 OUTPUT In.1, In.0 'sub'
  8   ~0%     {2} r10 = JOIN r9 WITH Expr::Expr::getUnspecifiedType_dispred#bb ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'sub'
  8   ~0%     {1} r11 = JOIN r10 WITH Type::IntegralType::isUnsigned_dispred#f ON FIRST 1 OUTPUT Lhs.1 'sub'
              return r11

Tuple counts for UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff/2@i2#61800weu after 1ms:
  8  ~0%      {2} r1 = JOIN UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff#prev_delta WITH UnsignedDifferenceExpressionComparedZero::interestingSubExpr#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'sub'

  0  ~0%      {2} r2 = JOIN r1 WITH DataFlowUtil::localFlowStep#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  1  ~0%      {2} r3 = JOIN r1 WITH DataFlowUtil::localFlowStep#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  0  ~0%      {3} r4 = JOIN UnsignedDifferenceExpressionComparedZero::isGuarded#fff#prev_delta WITH UnsignedDifferenceExpressionComparedZero::interestingSubExpr#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'sub', Lhs.2
  0  ~0%      {3} r5 = JOIN r4 WITH DataFlowUtil::TExprNode#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n', Lhs.2
  0  ~0%      {2} r6 = JOIN r5 WITH UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff#prev ON FIRST 2 OUTPUT Lhs.2, Lhs.0 'sub'
  0  ~0%      {2} r7 = JOIN r6 WITH DataFlowUtil::TExprNode#ff ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1 'n'

  1  ~0%      {2} r8 = r3 UNION r7
  1  ~0%      {2} r9 = r2 UNION r8
  ...
2022-01-10 17:28:14 +00:00
Mathias Vorreiter Pedersen
f2d6bcd767 C++: Fix join order in 'isGuarded'.
Before:

Tuple counts for UnsignedDifferenceExpressionComparedZero::isGuarded#bff/3@ec24001m after 1.7s:
  97431    ~0%     {2} r1 = JOIN UnsignedDifferenceExpressionComparedZero::isGuarded#bff#join_rhs WITH project#BasicBlocks::Cached::basic_block_member ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'sub'
  11809769 ~1%     {2} r2 = JOIN r1 WITH Guards::GuardCondition::controls_dispred#fff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'sub', Rhs.1
  11809769 ~0%     {4} r3 = JOIN r2 WITH project#BasicBlocks::Cached::basic_block_member ON FIRST 1 OUTPUT Lhs.1, Rhs.1, false, Lhs.0 'sub'
  629277   ~4%     {7} r4 = JOIN r3 WITH Guards::GuardCondition::ensuresLt_dispred#ffffff_045123#join_rhs ON FIRST 3 OUTPUT Lhs.3 'sub', Lhs.0, Lhs.1, false, Rhs.3 'left', Rhs.4 'right', Rhs.5
  628120   ~4%     {7} r5 = SELECT r4 ON In.6 >= 0
  628120   ~1%     {3} r6 = SCAN r5 OUTPUT In.0 'sub', In.4 'left', In.5 'right'
                    return r6

After:

Tuple counts for UnsignedDifferenceExpressionComparedZero::isGuarded#fff/3@i2#a5071x3a after 392ms:
  103763 ~0%     {2} r1 = SCAN UnsignedDifferenceExpressionComparedZero::exprIsSubLeftOrLess#ff#prev_delta OUTPUT In.0 'sub', 26
  103763 ~0%     {1} r2 = JOIN r1 WITH exprs ON FIRST 2 OUTPUT Lhs.0 'sub'
  97431  ~0%     {3} r3 = JOIN r2 WITH project#BasicBlocks::Cached::basic_block_member ON FIRST 1 OUTPUT Rhs.1, false, Lhs.0 'sub'
  629277 ~0%     {7} r4 = JOIN r3 WITH Guards::GuardCondition::ensuresLt_dispred#ffffff_450123#join_rhs ON FIRST 2 OUTPUT Lhs.2 'sub', Lhs.0, false, Rhs.2, Rhs.3 'left', Rhs.4 'right', Rhs.5
  628120 ~0%     {7} r5 = SELECT r4 ON In.6 >= 0
  628120 ~1%     {6} r6 = SCAN r5 OUTPUT In.0 'sub', In.1, In.3, In.4 'left', In.5 'right', In.6
  628120 ~1%     {6} r7 = r6 AND NOT UnsignedDifferenceExpressionComparedZero::isGuarded#fff#prev(Lhs.0 'sub', Lhs.3 'left', Lhs.4 'right')
  628120 ~0%     {5} r8 = SCAN r7 OUTPUT In.2, In.1, In.0 'sub', In.3 'left', In.4 'right'
  628120 ~1%     {3} r9 = JOIN r8 WITH Guards::GuardCondition::controls_dispred#fff ON FIRST 2 OUTPUT Lhs.2 'sub', Lhs.3 'left', Lhs.4 'right'
                  return r9
2022-01-10 17:03:40 +00:00
Tom Hvitved
fd60c6e1ad Merge pull request #7510 from github/release-prep/2.7.5
Release preparation for version 2.7.5
2022-01-04 18:57:43 +01:00
github-actions[bot]
1dfcf427aa Release preparation for version 2.7.5 2022-01-04 14:44:56 +00:00
Tom Hvitved
bfb573c86a Merge pull request #7508 from hvitved/python/change-note-typo2
Python: Fix another change note typo
2022-01-04 14:10:37 +01:00
Tom Hvitved
6b4eaf674f Python: Fix another change note typo 2022-01-04 13:53:07 +01:00
Erik Krogh Kristensen
b9964799f3 Merge pull request #7458 from erik-krogh/modelling
QL: add "modelling/modeling" to `ql/non-us-spelling`
2022-01-04 13:33:54 +01:00
Anders Schack-Mulligen
6457f42497 Merge pull request #7500 from zbazztian/stringbuilder-reverse-taint
Propagate taint through AbstractStringBuilder.reverse()
2022-01-04 13:28:14 +01:00
Geoffrey White
344e380fa3 Merge pull request #6949 from ihsinme/ihsinme-patch-073
CPP: Add query for CWE-266 Incorrect Privilege Assignment
2022-01-04 11:37:17 +00:00
Tom Hvitved
a2c1995b9b Merge pull request #7506 from hvitved/python/change-note-typo
Python: Fix typo in change note
2022-01-04 11:47:48 +01:00
Anders Schack-Mulligen
f8380dabe0 Update java/ql/lib/semmle/code/java/frameworks/Strings.qll 2022-01-04 11:47:26 +01:00
Tom Hvitved
23fb3455c0 Python: Fix typo in change note 2022-01-04 11:06:23 +01:00
Tom Hvitved
1f8a291d6f Merge pull request #7198 from hvitved/ruby/dataflow/arrays
Ruby: Flow through arrays/enumerables
2022-01-04 10:37:08 +01:00
yoff
5ba70ff3b6 Merge pull request #7369 from RasmusWL/filter-tag-cwe
JS/Py/Ruby: Add more CWEs to bad-tag-filter queries
2022-01-04 10:11:03 +01:00
Michael Nebel
c3007ff713 Merge pull request #7468 from michaelnebel/csharp-foreach-dataflow
C#: Re-factor the ForEachCapture query to use MaD flow summaries.
2022-01-04 09:46:39 +01:00
Tom Hvitved
de1697ab39 Merge pull request #7503 from dbartol/dbartol/move-change-notes
Move change notes to correct location
2022-01-04 09:35:21 +01:00
Dave Bartolomeo
5f5af4a29e Move change notes to correct location
A few change notes slipped through the cracks of my previous change. These are now in the proper locations: `old-change-notes` for older notes, and `<lang>\ql\[src|lib]\change-notes` for current change notes.
2022-01-03 18:21:16 -05:00
Dave Bartolomeo
ded3c52a34 Merge pull request #7407 from github/post-release-prep/codeql-cli-2.7.4
Post-release preparation for codeql-cli-2.7.4
2022-01-03 17:09:58 -05:00
github-actions[bot]
1334d207fa Post-release version bumps 2022-01-03 20:11:15 +00:00
Taus
ec533c8465 Merge pull request #7502 from tausbn/ql-support-trailing-comma-in-set-literals
QL: Support trailing comma in set literals
2022-01-03 17:06:46 +01:00
Taus
8845529548 QL: Support trailing comma in set literals
See

725395405e

for the grammar changes and corresponding test.
2022-01-03 15:48:24 +00:00
Sebastian Bauersfeld
421bd1b970 Propagate taint through AbstractStringBuilder.reverse() and its overrides. 2022-01-03 10:38:27 +07:00
Tom Hvitved
882caf4011 Merge pull request #7470 from hvitved/csharp/dispatch-join-order
C#: Fix bad join-order in dispatch library
2021-12-22 19:11:33 +01:00
Alex Ford
0cbf136e21 Merge pull request #7273 from github/ruby/crypto-algorithms
Ruby: add CryptoAlgorithms library
2021-12-22 17:42:59 +00:00
Alex Ford
69f1c18a39 Merge pull request #7446 from jeffgran/jg/constant-write-access
[Ruby] Bugfix: ConstantWriteAccess::getQualifiedName() returns wrong value in some cases
2021-12-22 17:07:49 +00:00
Alex Ford
3da98ecb73 Bump a date 2021-12-22 16:38:16 +00:00
Alex Ford
a2104de8a0 Move CryptoAlgorithms::AlgorithmsName into a separate internal/CryptoAlgorithmNames.qll 2021-12-22 16:38:15 +00:00
Alex Ford
f16d77615d Remove unused isStrongBlockMode predicate from CryptoAlgorithms.qll 2021-12-22 16:38:15 +00:00
Alex Ford
df0da980ea Update ruby/ql/lib/codeql/ruby/security/OpenSSL.qll
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-12-22 16:38:15 +00:00
Alex Ford
27a40fb5cf Ruby: OpenSSL QLDoc fixes 2021-12-22 16:38:15 +00:00
Alex Ford
97c75de771 Ruby: OpenSSL and CryptoAlgorithms test update 2021-12-22 16:38:15 +00:00
Alex Ford
e6bc45ee3b Ruby: Base OpenSSL supported algorithms on OpenSSL 1.1.1 and LibreSSL 3.4.1 2021-12-22 16:38:15 +00:00
Alex Ford
d3af687767 Add more encryption algorithms and modes to CryptoAlgorithms::AlgorithmNames
Strong encryption algorithms: ARIA, IDEA, SEED, SM4
Strong block modes: CBC, CFB, CTR, OFB
2021-12-22 16:38:15 +00:00
Alex Ford
bdb2d8ba16 Ruby: split OpenSSL parts from CryptoALgorithms.qll and sync with JS/Python version 2021-12-22 16:38:15 +00:00
Alex Ford
0303c279e2 Ruby: add empty ruby file to avoid DataFlowConsistency failure 2021-12-22 16:38:15 +00:00
Alex Ford
1156581b52 Ruby: add CryptoAlgorithms library 2021-12-22 16:38:15 +00:00
Jeff Gran
accfd482d4 autoformat file 2021-12-22 08:44:35 -07:00
Jeff Gran
6acb87d542 add change-notes 2021-12-22 08:42:07 -07:00
Jeff Gran
f21398ce84 changed the name of one of the constants for a better test case 2021-12-22 08:42:07 -07:00
Jeff Gran
445c420a3d rerun test --learn with rebuilt ruby extractor 2021-12-22 08:42:04 -07:00
Jeff Gran
07c7de5cfd run test --learn, add a few more constants to constant.rb test case 2021-12-22 08:36:07 -07:00
Jeff Gran
7c032f6cb4 fix docs, fix deprecations 2021-12-22 08:35:55 -07:00
Jeff Gran
f35e866799 Capitalize "Gets"
Co-authored-by: Arthur Baars <aibaars@github.com>
2021-12-22 08:35:55 -07:00
Jeff Gran
0c698996aa use resolveConstanteWriteAccess instead, add a few more test cases 2021-12-22 08:35:55 -07:00
Jeff Gran
3df7793803 add more test cases, fix bug by adding getFullName() predicate 2021-12-22 08:35:55 -07:00
Jeff Gran
8e46eeb88c fix expectations to expect the correct values 2021-12-22 08:35:52 -07:00
Tom Hvitved
55492ef348 Ruby: Update expected test output after rebase 2021-12-22 15:56:20 +01:00
Tom Hvitved
118d0d9ff5 Ruby: Use "Receiver" instead of "Self" in flow summaries
Flow summaries use the "outside view", i.e., the call sites, so "receiver"
is better than "self", as the latter uses the "inside view", i.e. the callees.
2021-12-22 15:56:20 +01:00
Tom Hvitved
3a30f58f74 Address review comments 2021-12-22 15:56:20 +01:00