Commit Graph

614 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
0f95992b2f Python: remove NonLibraryDataFlowCallable
this required managing parameters and their pre-update nodes a bit
2022-09-12 15:17:29 +02:00
Rasmus Lerchedahl Petersen
895f5480c2 Python: Added recursion guard
to ensure that the call graph seen by type tracking
does not include summary calls resolved by type tracking.

(I tried inserting a similar test into the Ruby codebase,
 and it still compiled)

To get this to compile, I had to move the resolution of summary calls
out of the data flow nodes and into the `viableCallable` predicate.
This means that we now have a potential summary call for each
cfg call node. (I tried using the base class, `DataFlowCall`, for this
but calls to `map` got identified as class calls and would no longer
be associated with a summary.)

It is possible that the "NonLIbrary"-layers the were inserted into the
hierarchy can be removed again.
2022-09-09 22:47:47 +02:00
Rasmus Lerchedahl Petersen
f6d807aec0 Python: Add summary test append_to_list 2022-09-06 18:42:32 +02:00
Rasmus Lerchedahl Petersen
4cd41c24c7 Python: remove comments and start design document 2022-09-06 17:23:40 +02:00
Rasmus Lerchedahl Petersen
67c3a9b2f4 Python: resolve library calls in the CFG
rather than in the AST
2022-09-06 17:00:28 +02:00
yoff
6b4716485b Python: rename file 2022-08-25 12:23:09 +00:00
yoff
800165d63c python: udate deprecated call 2022-08-25 09:49:46 +00:00
yoff
75ac24a847 Merge branch 'main' into python-dataflow/flow-summaries-from-scratch 2022-08-10 10:57:59 +02:00
Rasmus Wriedt Larsen
f89b32183f Merge branch 'main' into typetracker-decorators 2022-08-08 11:52:09 +02:00
yoff
f52d792b36 Merge branch 'main' of https://github.com/github/codeql into python-dataflow/flow-summaries-from-scratch 2022-07-01 12:01:07 +00:00
yoff
61523bd330 python: better names
- "Normal" instead of "NonSpecial"
- "NonLibrary" instead of "2"

I could not find a good replacement for "NonLibrary", nor for "Source",
but I added QLDocs in a few places to help the reading.
2022-07-01 11:55:20 +00:00
yoff
6087bc6888 Merge branch 'main' into python/more-logic-tests 2022-06-28 22:16:38 +02:00
Asger F
a522562f93 Merge pull request #9369 from asgerf/python/api-graph-api
Python: API graph renaming and documentation
2022-06-28 14:48:12 +02:00
Rasmus Lerchedahl Petersen
a1fe8a5b2b python: handle not in BarrierGuard
in the program
```python
if not is_safe(path):
  return
```
the last node in the `ConditionBlock` is `not is_safe(path)`,
so it would never match "a call to is_safe".
Thus, guards inside `not` would not be part of `GuardNode`
(nor `BarrierGuard`). Now they can.
2022-06-27 20:10:47 +00:00
Rasmus Lerchedahl Petersen
882000afb3 python: not is confusing our logic
- added `is_unsafe`
- added "negated version" of two tests.
These versions do not use `not` and the analysis gets the taint right.
2022-06-27 20:10:47 +00:00
Rasmus Wriedt Larsen
3248f7b423 Merge pull request #9649 from RasmusWL/certificate-modeling
Python/JS/Ruby: Ignore common words (like certain) as sensitive data source
2022-06-23 12:04:58 +02:00
yoff
140dc1a61e merge in main 2022-06-23 09:05:32 +00:00
yoff
8bf60301da python: we have hidden isParameterOf
but now allow a clear alternative
2022-06-23 08:57:50 +00:00
yoff
fe0c5d8ee5 python: make ArgumentNode publicly usable
- add `getCall`
2022-06-23 08:48:55 +00:00
yoff
cedf9ef538 python: make DataFlowCall "publicly usable"
- add `getCallable`, `getArg` and `getNode`
- these are `none` for summary calls
- revert "external" uses (they had been changed to `DataFlowSourceCall`)
2022-06-23 08:32:23 +00:00
Rasmus Wriedt Larsen
4be375521f Python: Handle _ in sensitive-data-sources 2022-06-22 11:05:14 +02:00
Rasmus Wriedt Larsen
4a844312f4 Python: _ in var name not handled by sensitive-data-sources 2022-06-22 11:05:14 +02:00
Rasmus Wriedt Larsen
5dc2bb717a Python: ignore common words (certain/concert) as sensitive source 2022-06-22 11:05:05 +02:00
Rasmus Wriedt Larsen
abdcfd55c3 Python: uncertainty is treated as a certificate :O 2022-06-22 10:16:28 +02:00
yoff
dd69100dcd python: ParameterNode -> SourceParameterNode 2022-06-21 12:55:22 +00:00
Asger F
181a53bd03 Python: Rename getAnImmediateUse -> asSource 2022-06-21 12:44:06 +02:00
Anders Schack-Mulligen
f473a0a961 Python: Deprecate and replace BarrierGuard class. 2022-06-20 15:46:38 +02:00
Rasmus Wriedt Larsen
b2c8e0fe8d Python: Add comment to test 2022-06-15 15:59:54 +02:00
Rasmus Wriedt Larsen
24c9aff2fc Python: Fix a type-tracking test 2022-06-15 15:58:17 +02:00
Tom Hvitved
4f95abc4f6 Python: Update expected test output 2022-05-25 14:39:37 +02:00
yoff
8b9915e372 Python: Let the user help us identifying callbacks 2022-05-23 11:07:47 +00:00
yoff
2822ed9594 Merge remote-tracking branch 'upstream/main' into python-dataflow/flow-summaries-from-scratch 2022-05-16 08:10:15 +00:00
Nick Rolfe
2efa38aaa6 Python: fix typos in comments 2022-05-12 16:02:20 +01:00
yoff
0778d90ac1 python: fix implementation of lambdaCreation
- still identifying summarized callables by name.
I think ther shoudl perhaps be a `getAUse` next to `getACall`.
- also fix tests, adding a standard taint configuration
2022-05-12 11:35:57 +00:00
Rasmus Wriedt Larsen
795adf0566 Python: Fix API::moduleImport("foo.bar") 2022-05-12 13:33:00 +02:00
yoff
da3634188d python: variaous fixes
- sync summary files
- format files
- fix compilation
2022-05-10 12:48:42 +00:00
yoff
f14ee0e794 python: Flow summaries based on type tracking
Two classes have been inserted into the hierarchies:

- `NonLibraryDataFlowCallable` with a method `getACall2`.
This method implements "get a call, not considering flow summaries".
For `NonLibraryDataFlowCallable`s, `getACall` will defer to `getACall2`.
While you could have a synthesised call to such a callable,
it would not correspond to a `CallNode`.

- `NonLibraryDataFlowSourceCall` with methods
`getArg2` and `getCallable2`. These also refer to a call graph that
does not consider flow summaries.

`getArg2` is used to synthesise pre-update nodes for arguments.

`getCallable2` is used in `connects` to compute argument passing.
This is used to define data flow nodes for overflow arguments.

`getACall2` ensures that `LibraryCallableValue::getACall` is not called
when the charpred of `FunctionCall` is evaluated.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
506efcf051 python: refactor TDataFlowCall
- Branch predicates are made simple. In particular, they do not try to detect library calls.
- All branches based on `CallNode`s are gathered into one.
- That branch has been given a class `NonSpecialCall`, which is the new parent of call classes based on `CallNode`s. (Those classes now have more involved charpreds.)
- A new such class, 'LambdaCall` has been split out from `FunctionCall` to allow the latter to replace its
  general `CallNode` field with a specific `FunctionValue` one.
- `NonSpecialCall` is not an abstract class, but it has some abstract overrides. Therefor, it is not
  considered a resolved call in the test `UnresolvedCalls.qll`.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
177dea5307 python: use new syntax for flow summaries
also convert to inline tests
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
4024ce4777 python: some summary flows 2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
828db3a392 python: Add summary nodes
allowing more `OutNode`s (not restricting to `CallNode`s),
gives more flow in the `classesCallGraph` test
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
80175a9af5 Python: Compiles and mostly pass tests
- add flowsummaries shared files
- register in indentical files
- fix initial non-monotonic recursions
  - add DataFlowSourceCall
  - add resolvedCall
  - add SourceParameterNode

failing tests:
- 3/library-tests/with/test.ql
2022-05-10 12:48:42 +00:00
yoff
1d44694280 Merge pull request #8732 from RasmusWL/dataflow-imports
Python: Don't re-export `python` under `DataFlow::`
2022-05-02 12:08:28 +02:00
Taus
231def026f Merge pull request #8890 from tausbn/python-add-global-attribute-writes
Python: Add support for global attribute writes
2022-05-02 12:03:41 +02:00
yoff
c67b06b1fd Update python/ql/test/experimental/dataflow/typetracking/attribute_tests.py
Co-authored-by: Taus <tausbn@github.com>
2022-05-02 11:36:58 +02:00
Taus
b4a31e572f Python: Add global attribute writes 2022-04-27 16:45:00 +00:00
Taus
f71cf2e1fc Python: Add test 2022-04-27 15:48:11 +00:00
Rasmus Wriedt Larsen
084c8eb22e Python: Don't re-export python under DataFlow:: 2022-04-20 11:42:10 +02:00
Rasmus Wriedt Larsen
5dbbd17bb2 Python: Add test to ensure we keep DataFlow imports clean
Currently we're not in a good state :(
2022-04-20 11:41:01 +02:00
Erik Krogh Kristensen
50bfc8eaa0 refactor uses of API::Node::getAUse() that should have been something else 2022-04-07 13:52:13 +02:00