Commit Graph

29 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
03c243175b Python: fix QL alerts 2022-09-12 23:53:42 +02:00
Rasmus Lerchedahl Petersen
efc5cfb852 Merge branch 'main' of github.com:github/codeql into python-dataflow/flow-summaries-from-scratch 2022-09-12 19:56:16 +02:00
Rasmus Lerchedahl Petersen
0f95992b2f Python: remove NonLibraryDataFlowCallable
this required managing parameters and their pre-update nodes a bit
2022-09-12 15:17:29 +02:00
erik-krogh
cc7a9ef97a rename more acronyms 2022-08-25 20:52:27 +02:00
yoff
61523bd330 python: better names
- "Normal" instead of "NonSpecial"
- "NonLibrary" instead of "2"

I could not find a good replacement for "NonLibrary", nor for "Source",
but I added QLDocs in a few places to help the reading.
2022-07-01 11:55:20 +00:00
yoff
fe0c5d8ee5 python: make ArgumentNode publicly usable
- add `getCall`
2022-06-23 08:48:55 +00:00
yoff
cedf9ef538 python: make DataFlowCall "publicly usable"
- add `getCallable`, `getArg` and `getNode`
- these are `none` for summary calls
- revert "external" uses (they had been changed to `DataFlowSourceCall`)
2022-06-23 08:32:23 +00:00
yoff
dd69100dcd python: ParameterNode -> SourceParameterNode 2022-06-21 12:55:22 +00:00
yoff
0778d90ac1 python: fix implementation of lambdaCreation
- still identifying summarized callables by name.
I think ther shoudl perhaps be a `getAUse` next to `getACall`.
- also fix tests, adding a standard taint configuration
2022-05-12 11:35:57 +00:00
yoff
da3634188d python: variaous fixes
- sync summary files
- format files
- fix compilation
2022-05-10 12:48:42 +00:00
yoff
f14ee0e794 python: Flow summaries based on type tracking
Two classes have been inserted into the hierarchies:

- `NonLibraryDataFlowCallable` with a method `getACall2`.
This method implements "get a call, not considering flow summaries".
For `NonLibraryDataFlowCallable`s, `getACall` will defer to `getACall2`.
While you could have a synthesised call to such a callable,
it would not correspond to a `CallNode`.

- `NonLibraryDataFlowSourceCall` with methods
`getArg2` and `getCallable2`. These also refer to a call graph that
does not consider flow summaries.

`getArg2` is used to synthesise pre-update nodes for arguments.

`getCallable2` is used in `connects` to compute argument passing.
This is used to define data flow nodes for overflow arguments.

`getACall2` ensures that `LibraryCallableValue::getACall` is not called
when the charpred of `FunctionCall` is evaluated.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
506efcf051 python: refactor TDataFlowCall
- Branch predicates are made simple. In particular, they do not try to detect library calls.
- All branches based on `CallNode`s are gathered into one.
- That branch has been given a class `NonSpecialCall`, which is the new parent of call classes based on `CallNode`s. (Those classes now have more involved charpreds.)
- A new such class, 'LambdaCall` has been split out from `FunctionCall` to allow the latter to replace its
  general `CallNode` field with a specific `FunctionValue` one.
- `NonSpecialCall` is not an abstract class, but it has some abstract overrides. Therefor, it is not
  considered a resolved call in the test `UnresolvedCalls.qll`.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
828db3a392 python: Add summary nodes
allowing more `OutNode`s (not restricting to `CallNode`s),
gives more flow in the `classesCallGraph` test
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
80175a9af5 Python: Compiles and mostly pass tests
- add flowsummaries shared files
- register in indentical files
- fix initial non-monotonic recursions
  - add DataFlowSourceCall
  - add resolvedCall
  - add SourceParameterNode

failing tests:
- 3/library-tests/with/test.ql
2022-05-10 12:48:42 +00:00
Taus
5a8ba6a7af Python: Fix use of singleton set 2022-03-07 18:59:49 +00:00
Rasmus Wriedt Larsen
999af15bd5 Python: Show unresolved calls for field-flow tests 2022-02-25 15:58:07 +01:00
Rasmus Wriedt Larsen
c5b6fb37b7 Python: Clean up NormalDataflowTest.qll 2022-02-18 13:47:29 +01:00
Rasmus Wriedt Larsen
5ee755db09 Python: Require MISSING: flow annotations for normal data-flow tests
I had to rewrite the SINK1-SINK7 definitions, since this new requirement
complained that we had to add this `MISSING: flow` annotation :D

Doing this implementation also revealed that there was a bug, since I
did not compare files when checking for these `MISSING:` annotations. So
fixed that up in the implementation for inline taint tests as well.

(extra whitespace in argumentPassing.py to avoid changing line numbers
for other tests)
2022-02-01 17:46:53 +01:00
Rasmus Wriedt Larsen
2bc4a60496 Python: Unify normal dataflow test setup
I went with NormalDataflowTest to signify that if you don't know what
you're looking for, this is probably the one. I did not want to just
call it DataflowTest, since that becomes a big vague when there are also
`FlowTest.qll` and `MaximalFlowTest.qll` -- I'm open to renaming this
though 👍
2022-02-01 17:31:31 +01:00
Erik Krogh Kristensen
6ff8d4de5c add all remaining explicit this 2021-11-26 13:50:10 +01:00
Rasmus Wriedt Larsen
ce4b192caa Python: Improve usefulness of RemoteFlowSourcesReach meta query
Before, results from `dca` would look something like

    ## + py/meta/alerts/remote-flow-sources-reach

    - django/django@c2250cf_cb8f: tests/messages_tests/urls.py:38:16:38:48
        reachable with taint-tracking from RemoteFlowSource
    - django/django@c2250cf_cb8f: tests/messages_tests/urls.py:38:9:38:12
        reachable with taint-tracking from RemoteFlowSource

now it should make it easier to spot _what_ it is that actually changed,
since we pretty-print the node.
2021-07-21 16:35:09 +02:00
Rasmus Wriedt Larsen
61ad5d0673 Python: Allow printing PostUpdateNode in ConceptsTest.qll
See how this works in `test_json.py`
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
f66dccafda Python: Rename prettyExp => prettyExpr
So we're consistenly using `expr` and not leaving our the `r`.
2021-05-19 17:10:33 +02:00
Rasmus Lerchedahl Petersen
e786be06ae Python: Fix broken references 2021-01-21 12:40:35 +01:00
Rasmus Lerchedahl Petersen
419449fb8a Python: default value for argN 2021-01-20 20:33:04 +01:00
Rasmus Lerchedahl Petersen
2409a7899b Python: Remove func tag in some situations.
Also make ArgumentNode public
2021-01-20 20:18:40 +01:00
yoff
3fc085ff38 Update python/ql/test/experimental/dataflow/TestUtil/RoutingTest.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-01-20 16:11:40 +01:00
Rasmus Lerchedahl Petersen
77da4b0106 Python: Remove absolute line numbers
- Use relative line numbers in flow test
- Elide line numbers in routing test (new concept)
2021-01-19 17:05:42 +01:00
Rasmus Lerchedahl Petersen
a19304a4a0 Python: Factor out prettyPrinter and update tests 2020-11-24 02:17:38 +01:00