Commit Graph

71148 Commits

Author SHA1 Message Date
Taus
eaef783f4b Python: Add partial model for copy.replace
Extends our modelling to partially cover the behaviour of
`copy.replace`. In particular, we model this in two ways:

Firstly, we extend the existing Models-as-Data row for `copy` and
`deepcopy` to also cover `replace`. This means that we treat the result
of `replace` as containing all of the fields of the original object.
This is somewhat _more_ than we want, as strictly speaking the fields
that are overwritten should _not_ propagate flow through the `replace`
call, but currently we don't have a good way of modelling this blocking
of flow.

Secondly, we add a flow summary that adds flow from named arguments of
the `replace` call to the corresponding fields on the base object. This
ensures that we at least have the new flow arising from the `replace`
call.

Note that the flow summary adds this flow for _all_ named arguments of
_all_ `replace` calls throughout the codebase. However, since any
particular `replace` call will only populate a subset of these (the
subset consisting of exactly those named arguments that are in that
particular call), this does not cause any unwanted crosstalk between
different `replace` calls.§
2024-10-15 12:14:19 +00:00
Taus
6f2cfa0ba8 Python: Update CopySummary to use getMaDRepresentation
Demonstrates the somewhat more ergonomic way to use
`getMaDRepresentation` when specifying summaries.

Note that this slightly extends the previous definition, in that
`DictionaryContentAny` is now _also_ propagated by a call to the
`.copy()` method, but I think this is correct.
2024-10-15 11:52:37 +00:00
Taus
ce914019c5 Python: Add getMaDRepresentation()
This adds a convenient way of getting the Models-as-Data representation
of a particular type of content. This avoids repeating the same
construction over and over in our various summaries. Currently this is
defined for all types of content except the captured variable content,
which to my knowledge doesn't have any representation in Models-as-Data.
2024-10-15 11:50:38 +00:00
Taus
e16405c675 Python: Add test for copy.replace
This test demonstrates the current state of affairs: that `copy.replace`
essentially blocks all flow of taint through it, because it has not been
modelled yet.
2024-10-15 11:48:43 +00:00
Ian Lynagh
24d98eef83 Merge pull request #17741 from github/igfoo/logverb
Kotlin: Log our verbosity level
2024-10-15 11:42:15 +01:00
Owen Mansel-Chan
1626af0ae1 Merge pull request #17748 from owen-mc/go/join-order-fix/data-flow-node-gettype
Go: Fix bad join order in `SummarizedParameterNode.gettype`
2024-10-15 10:14:38 +01:00
Geoffrey White
7d9a3e1399 Merge pull request #17756 from geoffw0/unusedvar6
Rust: More test cases for rust/unused-variable
2024-10-15 10:02:37 +01:00
Arthur Baars
508e8157b3 Merge pull request #17746 from github/aibaars/macro-pat-irrefutable
Rust: add MacroPat to isExhaustiveMatch
2024-10-15 10:45:06 +02:00
Geoffrey White
5e33d9a145 Rust: Further changes from code review. 2024-10-15 09:38:17 +01:00
Geoffrey White
8169ccd21e Apply suggestions from code review
Co-authored-by: Simon Friis Vindum <paldepind@github.com>
2024-10-15 09:36:09 +01:00
Simon Friis Vindum
cc1e33fb64 Merge pull request #17713 from hvitved/rust/ssa
Rust: Initial SSA implementation
2024-10-15 09:59:48 +02:00
Geoffrey White
57875554f4 Rust: Accept consistency failures in the new test cases. 2024-10-14 17:06:55 +01:00
Geoffrey White
e5885f65db Rust: Add more test cases for unused variables and unreachable code. 2024-10-14 16:25:18 +01:00
Geoffrey White
528641c55a Rust: Test spacing. 2024-10-14 16:25:17 +01:00
Geoffrey White
134539060d Rust: correct some 'spurious' annotations that are off by one line. 2024-10-14 16:24:44 +01:00
Geoffrey White
25b592fc89 Merge pull request #17744 from geoffw0/unusedvar5
Rust: Unused variable follow-up work
2024-10-14 16:20:52 +01:00
Taus
5115216c2d Merge pull request #17755 from github/python/more-adj
python: more adjustments...
2024-10-14 17:16:16 +02:00
yoff
da5e9ac18c python: more adjustments... 2024-10-14 14:54:33 +00:00
Edward Minnix III
ade5686e52 Merge pull request #17335 from egregius313/egregius313/go/dataflow/models/stdin
Go: Implement `stdin` models
2024-10-14 10:38:27 -04:00
Taus
96ea9507a4 Merge pull request #17754 from github/python/extractor-expectations
python: update extractor expectations
2024-10-14 16:35:06 +02:00
yoff
9d8d7ab237 python: update extractor expectations 2024-10-14 14:14:40 +00:00
Arthur Baars
ae7afa7aff Update rust/ql/lib/codeql/rust/controlflow/internal/Completion.qll
Co-authored-by: Simon Friis Vindum <simonfv@gmail.com>
2024-10-14 15:20:43 +02:00
Arthur Baars
867c8a7fa6 Merge branch 'main' into aibaars/macro-pat-irrefutable 2024-10-14 15:02:28 +02:00
Simon Friis Vindum
ef1592fe32 Merge pull request #17751 from paldepind/rust-rename-predicate
Rust: Rename isIrrefutablePattern to isExhaustiveMatch
2024-10-14 14:47:15 +02:00
Arthur Baars
493a68a232 Rust: add MacroPat to isExhaustiveMatch 2024-10-14 13:55:59 +02:00
Simon Friis Vindum
22261c1480 Rust: Rename isIrrefutablePattern to isExhaustiveMatch 2024-10-14 13:43:57 +02:00
Simon Friis Vindum
d0f978d937 Merge pull request #17749 from paldepind/rust-cfg-handle-question-mark
Rust: Handle question mark operator in CFG
2024-10-14 13:19:46 +02:00
Arthur Baars
de612963de Merge pull request #17750 from yoff/python/extractor-test-expectaionts
Python: adjust test expectations for extractor test
2024-10-14 12:54:15 +02:00
Rasmus Lerchedahl Petersen
3402a729d0 Python: adjust test expectations for extractor test 2024-10-14 12:36:56 +02:00
Simon Friis Vindum
e83f1d17bf Rust: Simplify return implementation in CFG 2024-10-14 11:40:35 +02:00
Simon Friis Vindum
8e4e663739 Rust: Handle question mark operator in CFG 2024-10-14 11:38:35 +02:00
Simon Friis Vindum
2f07f1c730 Rust: Abnormal completion in scrutinee does not continue to match arms 2024-10-14 11:32:22 +02:00
Simon Friis Vindum
23ee7b955c Rust: Add CFG tests for question mark operator and scrutinee with return 2024-10-14 11:27:32 +02:00
Owen Mansel-Chan
1456ec2119 Fix bad join order in SummarizedParameterNode.gettype
Specifically the disjunct for this.getPos() != -1. Running on
uber/aresdb, before we had this:

   2403   ~1%    {3} r6 = JOIN `DataFlowUtil::SummarizedParameterNode.getPos/0#dispred#70a2aab4` WITH `DataFlowPrivate::FlowSummaryNode.getSummarizedCallable/0#dispred#e79ea9be` ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Rhs.1
9149774   ~5%    {4}    | JOIN WITH `Types::SignatureType.getParameterType/1#dispred#2c11bb7b_102#join_rhs` ON FIRST 1 OUTPUT Lhs.2, Rhs.1, Lhs.1, Rhs.2
    923   ~9%    {2}    | JOIN WITH `Scopes::Callable.getType/0#dispred#55a0e6a2` ON FIRST 2 OUTPUT Lhs.2, Lhs.3

We add a binding pragma to make it not bind on this.getPos() until
necessary. After we have this:

   2403   ~0%    {3} r6 = JOIN `DataFlowUtil::SummarizedParameterNode.getPos/0#dispred#70a2aab4` WITH `DataFlowPrivate::FlowSummaryNode.getSummarizedCallable/0#dispred#e79ea9be` ON FIRST 1 OUTPUT Rhs.1, Lhs.0, Lhs.1
   2373   ~0%    {3}    | JOIN WITH `Scopes::Callable.getType/0#dispred#55a0e6a2` ON FIRST 1 OUTPUT Rhs.1, Lhs.2, Lhs.1
    923   ~9%    {2}    | JOIN WITH `Types::SignatureType.getParameterType/1#dispred#2c11bb7b` ON FIRST 2 OUTPUT Lhs.2, Rhs.2
2024-10-13 14:27:51 +01:00
Tom Hvitved
6290be2922 Rust: Initial SSA implementation 2024-10-11 19:39:28 +02:00
Tom Hvitved
b0ea175021 Rust: Add more variables tests 2024-10-11 19:36:04 +02:00
yoff
8f681154ce Merge pull request #17683 from yoff/python/flip-default-for-stdlib-extraction
Python: Flip default for stdlib extraction
2024-10-11 18:34:44 +02:00
Geoffrey White
c4256f21c7 Rust: Fix out-of-date spurious tags. 2024-10-11 17:18:41 +01:00
Paolo Tranquilli
5b57826915 Merge pull request #17659 from github/aibaars/rust-macros
Rust: add macro expansion to the extractor
2024-10-11 17:29:29 +02:00
Ian Lynagh
9f340279cb Kotlin: Accept test changes 2024-10-11 15:10:19 +01:00
yoff
2af60f1390 Merge pull request #17712 from yoff/python/re-finditer-match
Python: model that `re.finditer` returns an iterable of `re.Match` objects
2024-10-11 16:08:49 +02:00
Arthur Baars
fc298b23c9 Rust: address comment 2024-10-11 16:01:13 +02:00
Arthur Baars
5f45402cf7 Merge remote-tracking branch 'upstream/main' into aibaars/rust-macros 2024-10-11 15:40:07 +02:00
Rasmus Lerchedahl Petersen
5650694313 Python: update tests 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
e2eb08b543 Python: improve messaging 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
22588c9f85 Python: update ectractor version 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
c013d4a59c Python: update test expectations 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
4a291147e0 Python: only look for the py2 stdlib if we extract std lib 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
bb78c2a67e Python: update test expectations 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
e91efaa92e python: do not extract stdlib by default 2024-10-11 15:36:44 +02:00