github-actions[bot]
71ab804274
Release preparation for version 2.16.6
2024-03-25 16:58:08 +00:00
github-actions[bot]
aebe9f6992
Post-release preparation for codeql-cli-2.16.5
2024-03-18 12:16:26 +00:00
github-actions[bot]
0a6243d07b
Release preparation for version 2.16.5
2024-03-18 10:14:07 +00:00
github-actions[bot]
dc9092c9ec
Post-release preparation for codeql-cli-2.16.4
2024-03-06 22:19:33 +00:00
github-actions[bot]
2f058ffb4d
Release preparation for version 2.16.4
2024-03-06 20:56:51 +00:00
Angela P Wen
ce31f8641a
Revert "Release preparation for version 2.16.4"
2024-03-06 12:07:33 -08:00
github-actions[bot]
661e68dab5
Release preparation for version 2.16.4
2024-03-05 18:13:58 +00:00
Angela P Wen
967963a653
Revert "Release preparation for version 2.16.4"
2024-03-05 08:53:33 -08:00
github-actions[bot]
a67218a027
Release preparation for version 2.16.4
2024-03-04 17:42:08 +00:00
yoff
00e77a3ddb
Merge pull request #15720 from RasmusWL/nosql-precision
...
Python: Add precision to NoSQL query
2024-03-04 14:44:46 +01:00
yoff
569bb991d4
Merge pull request #15775 from RasmusWL/scope-consistency
...
Python: Add consistency check for `PhaseDependentFlow`
2024-03-04 14:43:13 +01:00
Rasmus Wriedt Larsen
fbf6727809
Python: Add change-note
2024-03-04 11:46:38 +01:00
Rasmus Wriedt Larsen
16cb6c2044
Python: Fix validTest expectations
...
Co-authored-by: yoff <lerchedahl@gmail.com >
2024-03-04 11:41:47 +01:00
Rasmus Wriedt Larsen
85a45b0155
Python: Fix comment
...
Co-authored-by: yoff <lerchedahl@gmail.com >
2024-03-04 11:40:17 +01:00
Rasmus Wriedt Larsen
d99a763ef7
Python: add change-note
2024-03-01 15:24:33 +01:00
Rasmus Wriedt Larsen
eeda4355f1
Python: Fix missing DictionaryElementContent
2024-03-01 15:21:13 +01:00
Rasmus Wriedt Larsen
30b7fadbb8
Python: Add test
2024-03-01 15:19:56 +01:00
Rasmus Wriedt Larsen
d182eae868
Python: Add consistency check for PhaseDependentFlow
...
This would have found the problem in
https://github.com/github/codeql/pull/15755 .
As highlighted in the comment in the code, it's not a perfect solution
since we don't have an automatic way to ensure we don't introduce a new
PhaseDependentFlow use with a new step relation and forget to add it to
this consistency check... but I think this consistency check still adds
value!
2024-03-01 10:01:08 +01:00
Rasmus Wriedt Larsen
8079788a5f
Python: Add change-note
2024-02-28 16:44:25 +01:00
Rasmus Wriedt Larsen
cdf4dd16f0
Python: Fix module level flow for iterable unpacking
...
(and for * patterns in match)
Since `PhaseDependentFlow` uses the following predicate, that relies on
.getScope() to be present for there to be any importTimeFlow (flow at
toplevel scope), it's important that data-flow nodes implement `.getScope`.
```
private predicate isTopLevel(Node node) { node.getScope() instanceof Module }
```
By implementing getScope, we can now rely on default implementation of
`getEnclosingCallable` in DataFlow::Node:
```
/** Gets the enclosing callable of this node. */
DataFlowCallable getEnclosingCallable() { result = getCallableScope(this.getScope()) }
```
2024-02-28 16:39:08 +01:00
Rasmus Wriedt Larsen
e4699e092d
Python: Add test for iterable unpacking on module level
...
Currently doesn't work :O
2024-02-28 16:10:29 +01:00
Rasmus Wriedt Larsen
1cfac50749
Python: Add precision to NoSQL query
...
Due to this, it was not part of any query suite :O
2024-02-26 11:23:43 +01:00
Tom Hvitved
62b16c0fa3
Share getFileBySourceArchiveName implementation
2024-02-23 11:25:49 +01:00
Chris Smowton
f2e04c0cb2
Merge pull request #15672 from github/post-release-prep/codeql-cli-2.16.3
...
Post-release preparation for codeql-cli-2.16.3
2024-02-20 21:59:55 +00:00
yoff
d3ee5f65db
Merge pull request #15550 from yoff/python/remove-pointsto-from-module-getAnExport
...
python: remove a use of points-to
2024-02-20 19:04:46 +01:00
github-actions[bot]
37f8fa3413
Post-release preparation for codeql-cli-2.16.3
2024-02-20 16:50:47 +00:00
github-actions[bot]
6d061fbc35
Release preparation for version 2.16.3
2024-02-20 14:26:23 +00:00
Rasmus Lerchedahl Petersen
22e72d2fed
python: Move the rewrite out to Scope.qll
2024-02-20 10:39:29 +01:00
Rasmus Lerchedahl Petersen
de727bf1b5
Revert "python: remove a use of points-to"
...
This reverts commit 5cb71ce7e5
2024-02-20 10:23:31 +01:00
amammad
09d8a75844
Fix QLDoc issues
2024-02-14 23:31:22 +04:00
Rasmus Wriedt Larsen
eb401a205d
Python: Fix test exclusion for stdlib Python 3.12
2024-02-14 16:53:19 +01:00
Rasmus Wriedt Larsen
59014787a1
Python: Fix DataflowQueryTest
...
You're only allowed to have `result=OK` if there is a sink on that line...
2024-02-14 15:44:40 +01:00
Rasmus Wriedt Larsen
cd596f5d05
Python: Reformat test-file
...
All those newlines are not good for inline expectations
2024-02-14 15:44:06 +01:00
Rasmus Wriedt Larsen
e5bd633028
Python: Change name/id to Decompression Bomb
...
The old title/id matches how we used to write queries, but I think just
using the normal conversational name is easier for everyone :)
2024-02-14 14:54:25 +01:00
Rasmus Wriedt Larsen
69c8ef9898
Python: Use dataflow instead of taint-tracking
2024-02-14 14:52:37 +01:00
Rasmus Wriedt Larsen
ba7dd38fc9
Python: Delete duplicated file
2024-02-14 14:48:37 +01:00
Rasmus Wriedt Larsen
9ae3ea81ff
Python: Remove spurious results in stdlib
2024-02-14 14:47:28 +01:00
Rasmus Wriedt Larsen
d8fd457310
Python: Use helper predicate
...
Since the helper predicate had nice qldocs
2024-02-14 14:47:28 +01:00
Rasmus Wriedt Larsen
e7772f1062
Python: Use Unit class
2024-02-14 14:47:28 +01:00
Rasmus Wriedt Larsen
ad39b8c68b
Python: Accept .expected changes
2024-02-14 14:46:33 +01:00
Rasmus Wriedt Larsen
9399258e3b
Merge branch 'main' into amammad-python-bombs
2024-02-14 13:37:59 +01:00
Cornelius Riemenschneider
e08946ec8d
Merge pull request #15586 from github/criemen/bazel-python-language-pack
...
Python: Update BUILD.bazel files.
2024-02-13 10:19:12 +01:00
Cornelius Riemenschneider
27ebebc24b
Python: Update BUILD.bazel files.
...
This allows us to (later) build the whole python language pack with
bazel.
2024-02-12 17:10:35 +01:00
Rasmus Wriedt Larsen
cbb9a64bbb
Merge pull request #15457 from RasmusWL/psycopg
...
Python: Model the `psycopg` package
2024-02-12 15:59:16 +01:00
Tom Hvitved
1ea7717714
Capture flow: Take overwrites in nested scopes into account
2024-02-09 14:49:23 +01:00
Anders Schack-Mulligen
088a0a54ba
Python: Add empty provenance column to expected files.
2024-02-09 11:32:08 +01:00
Anders Schack-Mulligen
817aa7655f
Python: Remove redundant IncludePostUpdateFlow and PhaseDependentFlow application.
2024-02-09 11:32:08 +01:00
Dave Bartolomeo
92bd550c55
Merge pull request #15531 from github/post-release-prep/codeql-cli-2.16.2
...
Post-release preparation for codeql-cli-2.16.2
2024-02-08 05:58:17 -08:00
Rasmus Lerchedahl Petersen
5cb71ce7e5
python: remove a use of points-to
...
This is used by `Scope::isPublic` which in turn is called by the framework model for `setuptools`.
On my current quesry, this had a dramatic effect on the most expensive predicates:
Before
```
Most expensive predicates for completed query FindUses.ql:
time | evals | max @ iter | predicate
------|-------|--------------|----------
1m9s | 2933 | 123ms @ 422 | PointsTo::Expressions::equalityEvaluatesTo/4#ebe72212@cab7d3xr
43.1s | | | FlowSummaryImpl::Private::Steps::summaryLocalStep/3#900fb25e#ffb@8aa78a38
41.3s | 2936 | 2.1s @ 409 | PointsTo::InterProceduralPointsTo::scope_entry_value_transfer_from_earlier/4#acb2199d@cab7ddxr
30.2s | 2946 | 67ms @ 847 | PointsTo::PointsToInternal::multi_assignment_points_to/4#28782e93@cab7d0yr
29.7s | 2930 | 1.9s @ 30 | Extensions::ReModulePointToExtension.pointsTo_helper/1#a84effde@cab7dn4w
24.9s | 2933 | 84ms @ 414 | PointsTo::Expressions::inequalityEvaluatesTo/4#f0ecfab4@cab7d2xr
17.9s | 2582 | 306ms @ 31 | MRO::ClassListList.getItem/1#b6c27115#reorder_2_0_1@cab7dw6r
9.4s | 661 | 991ms @ 1 | SsaCompute::AdjacentUses::varBlockReaches/3#1824ad86@2b6af692
9.2s | 2738 | 26ms @ 664 | MRO::ClassList.containsSpecial/0#c967dabb#fb@cab7dg4w
8.9s | 2946 | 12ms @ 917 | PointsTo::Types::getBase/2#0ab04984@cab7du1w
7.4s | 2946 | 287ms @ 3 | PointsTo::PointsToInternal::points_to_candidate/4#0a587a42@cab7d80w
7.1s | 2934 | 14ms @ 2 | Constants::ConstantObjectInternal.attribute/3#6d9e12fc@cab7d6zr
6.8s | 2946 | 9ms @ 48 | PointsTo::InterProceduralPointsTo::callsite_points_to/4#72419c70@cab7dqxr
6.6s | 234 | 341ms @ 17 | ApiGraphs::API::Impl::rhs/3#2255afc6@a41b31w3
6.6s | 2946 | 86ms @ 5 | PointsTo::Types::six_add_metaclass/4#f926a4cb@cab7da0w
6.2s | 2930 | 341ms @ 30 | Extensions::RangeIterationVariableFact.pointsTo/3#662720c9#cpe#124@cab7di2w
5.9s | 287 | 61ms @ 4 | DataFlowDispatch::TrackAttrReadInput::start/2#67f26627@cc7b56yn
5.8s | | | DataFlowImplCommon::LambdaFlow::viableParamNonLambda/3#3123cc52_201#join_rhs@415f35h0
5.6s | | | FlowSummaryImpl::Private::Steps::viableParam/4#49c13ab8@2c1fcdq1
5.3s | | | FlowSummaryImpl::Private::Steps::viableParam/4#49c13ab8@22590ca9
5.2s | 233 | 276ms @ 21 | ApiGraphs::API::Impl::use/3#e6c88b66@a41b30w3
5.1s | 2945 | 177ms @ 4 | PointsTo::PointsToInternal::pointsTo/4#d99f16c6@cab7dj0w
4.7s | | | Flow::ControlFlowNode.toString/0#dispred#e1af144b@410c23a7
4.6s | 277 | 2.2s @ 6 | DataFlowDispatch::getCallArg/5#21589076@cc7b5vxn
4.5s | | | DataFlowImplCommon::Cached::viableParam/3#61239ead@cc05a1fv
4.3s | | | DataFlowImplCommon::LambdaFlow::viableParamNonLambda/3#3123cc52@cb992b2h
4.1s | | | _AstExtended::AstNode.getLocation/0#dispred#6b4dcb62_10#join_rhs_DataFlowPublic::Node.getLocation/0#__#shared@6ae639js
4s | | | Files::Location.toString/0#dispred#7e7e0516@b72abbo2
3.7s | | | locations_ast_234501#join_rhs@0859685o
3.7s | 10 | 1.7s @ 1 | ObjectInternal::ObjectInternal.toString/0#dispred#0b2e9429@6e8a4yh7
3.6s | 2942 | 63ms @ 94 | PointsTo::InterProceduralPointsTo::call_points_to_from_callee/4#394022a8@cab7d90w
3.6s | 232 | 213ms @ 18 | ApiGraphs::API::Impl::trackDefNode/2#8e3c4e6d@a41b33w3
3.6s | 2933 | 7ms @ 884 | PointsTo::Types::getInheritedMetaclass/2#097d39df#bff@cab7dr1w
3.6s | 2946 | 1.3s @ 13 | PointsTo::PointsToInternal::ssa_node_refinement_points_to/4#8ea6486b@cab7dnxr
3.5s | 1319 | 387ms @ 3 | SsaCompute::SsaDefinitions::reachesEndOfBlock/4#214bd902@fce54web
3.5s | 1320 | 385ms @ 2 | SsaCompute::SsaDefinitions::reachesEndOfBlockRec/4#63bb2cd4@fce54xeb
3.4s | 4861 | 478ms @ 2 | SsaCompute::SsaComputeImpl::ssaDefReachesRank/4#f19c6fee@cc8515rd
3.3s | | | _AstExtended::AstNode.getLocation/0#dispred#6b4dcb62_10#join_rhs_DataFlowPublic::Node.getLocation/0#__#higher_order_body@47ba63n6
3.3s | | | DataFlowPublic::Node.toString/0#dispred#af9c307a@4d16e7m6
3.3s | 2946 | 28ms @ 3 | PointsTo::PointsToInternal::reachableEdge/3#d3f53c12@cab7do7w
2.9s | 233 | 110ms @ 19 | ApiGraphs::API::Impl::trackUseNode/2#a0b4384d@a41b32w3
2.8s | 31 | 2.2s @ 9 | _Class::Class.getAMethod/0#dispred#66416e47_DataFlowDispatch::findFunctionAccordingToMroKnownStartin__#antijoin_rhs@L6#cc7b5
2.8s | 2737 | 21ms @ 444 | MRO::ClassListList.removedClassParts/4#de59b06f#reorder_2_3_4_0_1@cab7d06w
2.8s | 1322 | 462ms @ 4 | SsaCompute::Liveness::liveAtExit/2#b6aa63f4@6fd4cx73
2.8s | 2946 | 187ms @ 5 | PointsTo::Expressions::builtinCallPointsTo/5#3aa7f48b@cab7dwwr
2.8s | 2939 | 41ms @ 7 | PointsTo::PointsToInternal::use_points_to/4#ff1d0edd@cab7df0w
2.7s | 2946 | 20ms @ 92 | PointsTo::Conditionals::evaluates/5#736734b2#fbffff#reorder_5_0_2_1_3_4@cab7dp5w
2.6s | 2946 | 152ms @ 5 | Constants::callToBool/2#0b9b1e8d@cab7dn7w
2.5s | 287 | 24ms @ 4 | DataFlowDispatch::resolveClassInstanceCall/3#6e09c292@cc7b53xn
2.4s | 2946 | 31ms @ 5 | PointsTo::AttributePointsTo::variableAttributePointsTo/5#60adcc49@cab7dpwr
[2024-02-08 10:44:37] Total evaluation times for this run:
* Wall-clock duration of evaluation run: 1231.1 seconds
* Total time spent evaluating predicates: 1167.1 seconds
```
After
```
Most expensive predicates for completed query FindUses.ql:
time | evals | max @ iter | predicate
------|-------|--------------|----------
41.6s | | | FlowSummaryImpl::Private::Steps::summaryLocalStep/3#900fb25e#ffb@85aaaac1
9.2s | 661 | 905ms @ 1 | SsaCompute::AdjacentUses::varBlockReaches/3#1824ad86@2b6af692
7.6s | 234 | 502ms @ 19 | ApiGraphs::API::Impl::rhs/3#2255afc6@ce6d11wc
6.7s | | | DataFlowImplCommon::LambdaFlow::viableParamNonLambda/3#3123cc52_201#join_rhs@fd1dc5mi
6s | 287 | 80ms @ 113 | DataFlowDispatch::TrackAttrReadInput::start/2#67f26627@925826yr
5.7s | | | FlowSummaryImpl::Private::Steps::viableParam/4#49c13ab8@851052bl
5.6s | 233 | 289ms @ 21 | ApiGraphs::API::Impl::use/3#e6c88b66@ce6d10wc
5.4s | | | FlowSummaryImpl::Private::Steps::viableParam/4#49c13ab8@f2c42d17
4.8s | 277 | 2.4s @ 6 | DataFlowDispatch::getCallArg/5#21589076@92582vxr
4.7s | | | DataFlowImplCommon::Cached::viableParam/3#61239ead@ac08e0nf
4.7s | | | DataFlowImplCommon::LambdaFlow::viableParamNonLambda/3#3123cc52@82ff50ql
4.6s | | | Files::Location.toString/0#dispred#7e7e0516@b72abbo2
4.3s | | | Flow::ControlFlowNode.toString/0#dispred#e1af144b@410c23a7
4.2s | 232 | 249ms @ 19 | ApiGraphs::API::Impl::trackDefNode/2#8e3c4e6d@ce6d13wc
3.8s | | | _AstExtended::AstNode.getLocation/0#dispred#6b4dcb62_10#join_rhs_DataFlowPublic::Node.getLocation/0#__#shared@0ac73425
3.6s | 1319 | 354ms @ 1 | SsaCompute::SsaDefinitions::reachesEndOfBlock/4#214bd902@fce54web
3.6s | 1320 | 381ms @ 2 | SsaCompute::SsaDefinitions::reachesEndOfBlockRec/4#63bb2cd4@fce54xeb
3.4s | | | _AstExtended::AstNode.getLocation/0#dispred#6b4dcb62_10#join_rhs_DataFlowPublic::Node.getLocation/0#__#higher_order_body@9e946ea8
3.4s | 4861 | 474ms @ 2 | SsaCompute::SsaComputeImpl::ssaDefReachesRank/4#f19c6fee@cc8515rd
3.1s | 31 | 2.5s @ 9 | _Class::Class.getAMethod/0#dispred#66416e47_DataFlowDispatch::findFunctionAccordingToMroKnownStartin__#antijoin_rhs@L6#92582
3s | 53 | 114ms @ 48 | DataFlowDispatch::TrackAttrReadInput::start/2#67f26627@9ab38jw0
3s | 233 | 126ms @ 20 | ApiGraphs::API::Impl::trackUseNode/2#a0b4384d@ce6d12wc
3s | | | locations_ast_234501#join_rhs@0859685o
3s | | | DataFlowPublic::Node.toString/0#dispred#af9c307a@a2145cqf
2.8s | 234 | 206ms @ 21 | _ApiGraphs::API::Impl::MkDef#51c2f877#prev_ApiGraphs::API::Impl::trackDefNode/1#7e78e336#prev_delta___#antijoin_rhs#1@L9#ce6d1
2.8s | 1322 | 447ms @ 4 | SsaCompute::Liveness::liveAtExit/2#b6aa63f4@6fd4cx73
2.7s | 230 | 176ms @ 28 | ApiGraphs::API::Impl::MkDef#51c2f877@ce6d1w9c
2.5s | 287 | 50ms @ 112 | DataFlowDispatch::resolveClassInstanceCall/3#6e09c292@925823xr
2.4s | 234 | 246ms @ 19 | _ApiGraphs::API::Impl::MkDef#51c2f877#prev_ApiGraphs::API::Impl::trackDefNode/1#7e78e336#prev_delta___#antijoin_rhs@L4#ce6d1
2.3s | | | TaintTrackingPrivate::localAdditionalTaintStep/2#a2ec8c9d@e31201hd
2.2s | 53 | 72ms @ 15 | DataFlowDispatch::TrackAttrReadInput::start/2#67f26627@96b28jwo
2.2s | | | SensitiveDataSources::SensitiveDataModeling::sensitiveString/1#fdc3ad40@41f6ee2g
2s | | | DataFlowImplCommon::Cached::viableParamArg/3#4c55eddb@8f7f25oq
2s | | | Flow::ControlFlowNode.getExprChild/1#e757d179#bbf@db51e8ed
1.9s | | | project#FlowSummaryImpl::Private::Steps::viableParam/4#49c13ab8#2@e36c2dr8
1.9s | | | DataFlowPublic::Node.hasLocationInfo/5#dispred#b79d995f@6e929dfv
1.7s | 15 | 433ms @ 1 | PoorMansFunctionResolution::poorMansFunctionTracker/2#75430e01@e5202dnv
1.7s | | | #ImportResolution::ImportResolution::allowedEssaImportStep/2#f4117c61Plus#swapped@60d9daea
1.7s | 29 | 633ms @ 6 | _Class::Class.getAMethod/0#dispred#66416e47_Function::Function.getName/0#dispred#033700ef_10#join_rh__#antijoin_rhs@L4#92582
1.5s | 233 | 79ms @ 24 | ApiGraphs::API::Impl::trackUseNode/1#1af3a9ea@ce6d16wc
1.5s | | | ApiGraphs::API::Impl::edge/3#8453bf65@1bd8a6ja
1.5s | | | ApiGraphs::API::Node.getAValueReachableFromSource/0#dispred#9a406fb1@5dbb806u
1.3s | 1323 | 178ms @ 13 | SsaCompute::Liveness::liveAtEntry/2#bab3ea7c@6fd4cw73
1.3s | | | SsaCompute::SsaComputeImpl::defUseRank/4#782a2f48@0f27919s
1.3s | | | DataFlowDispatch::LibraryCallable.getACall/0#dispred#66a01171#fb@96b65frd
1.3s | | | ApiGraphs::API::Node.getAValueReachableFromSource/0#dispred#9a406fb1_10#join_rhs@c1dd43nv
1.3s | | | FlowSummaryImpl::Private::SummaryNode.toString/0#dispred#d499e234@63bd684g
1.2s | | | DataFlowDispatch::LibraryCallable.getACall/0#dispred#66a01171#fb@eaebb27g
1.2s | | | _DataFlowPublic::Node#da3b6093_DataFlowPublic::Node.asExpr/0#dispred#2845197a_py_exprs#antijoin_rhs@fcd8c3kj
1.2s | | | #ImportResolution::ImportResolution::allowedEssaImportStep/2#f4117c61Plus#swapped@c3f634us
[2024-02-08 11:43:50] Total evaluation times for this run:
* Wall-clock duration of evaluation run: 636.9 seconds
* Total time spent evaluating predicates: 562.4 seconds
```
2024-02-08 12:20:56 +01:00
github-actions[bot]
b5139078d0
Post-release preparation for codeql-cli-2.16.2
2024-02-06 19:22:35 +00:00
