Commit Graph

699 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
3248f7b423 Merge pull request #9649 from RasmusWL/certificate-modeling
Python/JS/Ruby: Ignore common words (like certain) as sensitive data source
2022-06-23 12:04:58 +02:00
yoff
140dc1a61e merge in main 2022-06-23 09:05:32 +00:00
yoff
8bf60301da python: we have hidden isParameterOf
but now allow a clear alternative
2022-06-23 08:57:50 +00:00
yoff
fe0c5d8ee5 python: make ArgumentNode publicly usable
- add `getCall`
2022-06-23 08:48:55 +00:00
yoff
cedf9ef538 python: make DataFlowCall "publicly usable"
- add `getCallable`, `getArg` and `getNode`
- these are `none` for summary calls
- revert "external" uses (they had been changed to `DataFlowSourceCall`)
2022-06-23 08:32:23 +00:00
Rasmus Wriedt Larsen
4be375521f Python: Handle _ in sensitive-data-sources 2022-06-22 11:05:14 +02:00
Rasmus Wriedt Larsen
4a844312f4 Python: _ in var name not handled by sensitive-data-sources 2022-06-22 11:05:14 +02:00
Rasmus Wriedt Larsen
5dc2bb717a Python: ignore common words (certain/concert) as sensitive source 2022-06-22 11:05:05 +02:00
Rasmus Wriedt Larsen
abdcfd55c3 Python: uncertainty is treated as a certificate :O 2022-06-22 10:16:28 +02:00
yoff
dd69100dcd python: ParameterNode -> SourceParameterNode 2022-06-21 12:55:22 +00:00
Asger F
181a53bd03 Python: Rename getAnImmediateUse -> asSource 2022-06-21 12:44:06 +02:00
Anders Schack-Mulligen
f473a0a961 Python: Deprecate and replace BarrierGuard class. 2022-06-20 15:46:38 +02:00
Rasmus Wriedt Larsen
b2c8e0fe8d Python: Add comment to test 2022-06-15 15:59:54 +02:00
Rasmus Wriedt Larsen
24c9aff2fc Python: Fix a type-tracking test 2022-06-15 15:58:17 +02:00
Tom Hvitved
4f95abc4f6 Python: Update expected test output 2022-05-25 14:39:37 +02:00
yoff
8b9915e372 Python: Let the user help us identifying callbacks 2022-05-23 11:07:47 +00:00
yoff
2822ed9594 Merge remote-tracking branch 'upstream/main' into python-dataflow/flow-summaries-from-scratch 2022-05-16 08:10:15 +00:00
Nick Rolfe
2efa38aaa6 Python: fix typos in comments 2022-05-12 16:02:20 +01:00
yoff
0778d90ac1 python: fix implementation of lambdaCreation
- still identifying summarized callables by name.
I think ther shoudl perhaps be a `getAUse` next to `getACall`.
- also fix tests, adding a standard taint configuration
2022-05-12 11:35:57 +00:00
Rasmus Wriedt Larsen
795adf0566 Python: Fix API::moduleImport("foo.bar") 2022-05-12 13:33:00 +02:00
yoff
da3634188d python: variaous fixes
- sync summary files
- format files
- fix compilation
2022-05-10 12:48:42 +00:00
yoff
f14ee0e794 python: Flow summaries based on type tracking
Two classes have been inserted into the hierarchies:

- `NonLibraryDataFlowCallable` with a method `getACall2`.
This method implements "get a call, not considering flow summaries".
For `NonLibraryDataFlowCallable`s, `getACall` will defer to `getACall2`.
While you could have a synthesised call to such a callable,
it would not correspond to a `CallNode`.

- `NonLibraryDataFlowSourceCall` with methods
`getArg2` and `getCallable2`. These also refer to a call graph that
does not consider flow summaries.

`getArg2` is used to synthesise pre-update nodes for arguments.

`getCallable2` is used in `connects` to compute argument passing.
This is used to define data flow nodes for overflow arguments.

`getACall2` ensures that `LibraryCallableValue::getACall` is not called
when the charpred of `FunctionCall` is evaluated.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
506efcf051 python: refactor TDataFlowCall
- Branch predicates are made simple. In particular, they do not try to detect library calls.
- All branches based on `CallNode`s are gathered into one.
- That branch has been given a class `NonSpecialCall`, which is the new parent of call classes based on `CallNode`s. (Those classes now have more involved charpreds.)
- A new such class, 'LambdaCall` has been split out from `FunctionCall` to allow the latter to replace its
  general `CallNode` field with a specific `FunctionValue` one.
- `NonSpecialCall` is not an abstract class, but it has some abstract overrides. Therefor, it is not
  considered a resolved call in the test `UnresolvedCalls.qll`.
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
177dea5307 python: use new syntax for flow summaries
also convert to inline tests
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
4024ce4777 python: some summary flows 2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
828db3a392 python: Add summary nodes
allowing more `OutNode`s (not restricting to `CallNode`s),
gives more flow in the `classesCallGraph` test
2022-05-10 12:48:42 +00:00
Rasmus Lerchedahl Petersen
80175a9af5 Python: Compiles and mostly pass tests
- add flowsummaries shared files
- register in indentical files
- fix initial non-monotonic recursions
  - add DataFlowSourceCall
  - add resolvedCall
  - add SourceParameterNode

failing tests:
- 3/library-tests/with/test.ql
2022-05-10 12:48:42 +00:00
yoff
1d44694280 Merge pull request #8732 from RasmusWL/dataflow-imports
Python: Don't re-export `python` under `DataFlow::`
2022-05-02 12:08:28 +02:00
Taus
231def026f Merge pull request #8890 from tausbn/python-add-global-attribute-writes
Python: Add support for global attribute writes
2022-05-02 12:03:41 +02:00
yoff
c67b06b1fd Update python/ql/test/experimental/dataflow/typetracking/attribute_tests.py
Co-authored-by: Taus <tausbn@github.com>
2022-05-02 11:36:58 +02:00
Taus
b4a31e572f Python: Add global attribute writes 2022-04-27 16:45:00 +00:00
Taus
f71cf2e1fc Python: Add test 2022-04-27 15:48:11 +00:00
Rasmus Wriedt Larsen
084c8eb22e Python: Don't re-export python under DataFlow:: 2022-04-20 11:42:10 +02:00
Rasmus Wriedt Larsen
5dbbd17bb2 Python: Add test to ensure we keep DataFlow imports clean
Currently we're not in a good state :(
2022-04-20 11:41:01 +02:00
Erik Krogh Kristensen
50bfc8eaa0 refactor uses of API::Node::getAUse() that should have been something else 2022-04-07 13:52:13 +02:00
yoff
5efc19c39d Merge pull request #7806 from erik-krogh/pyDef
Python: Add def nodes to API graphs
2022-03-28 08:09:14 +02:00
Taus
5a8ba6a7af Python: Fix use of singleton set 2022-03-07 18:59:49 +00:00
yoff
d953382df9 Merge pull request #7807 from RasmusWL/dataflow-improvements
Python: Dataflow improvements
2022-02-28 16:24:00 +01:00
Rasmus Wriedt Larsen
0e0f159891 Python: Use Python 3 for investigation tests
Apparently there are minor differences with `test-6-max-import-depth-2`
where under Python 2 `isfile_no_problem.py` still works as before
2022-02-28 11:33:31 +01:00
Rasmus Wriedt Larsen
01d426dc58 Python: Replace rest of from testlib import *
I think we should write our tests in a way that puts points-to in the
best condition to resolve calls. Although this specific change did not
change much, it should help set us up for success in the future 👍
2022-02-28 10:58:44 +01:00
Rasmus Wriedt Larsen
ead0b658d2 Python: Fix fieldflow tests by increasing max-import-depth 2022-02-28 10:58:44 +01:00
Rasmus Wriedt Larsen
a0d1cea490 Python: Add investigation of field-flow problem
TL;DR; we used a too low value for `--max-import-depth` :(
2022-02-28 10:58:44 +01:00
Rasmus Wriedt Larsen
f988e1f0d8 Python: Improve field-flow by removing import *
Since that apparently impacts call graph resolution with points-to :O

Also interesting that global flow was only not working for those cases
because of the tricky ifs... still need to 100% figure out how those ifs
are messing up the analysis :|
2022-02-25 16:01:08 +01:00
Rasmus Wriedt Larsen
999af15bd5 Python: Show unresolved calls for field-flow tests 2022-02-25 15:58:07 +01:00
yoff
8b926f6859 Merge pull request #7873 from RasmusWL/fix-attribute-taint
Python: Fix attribute taint
2022-02-25 15:02:24 +01:00
Rasmus Wriedt Larsen
2d0034c40d Python: Replicate global field-flow failures 2022-02-25 14:14:00 +01:00
Rasmus Wriedt Larsen
faaa63a73c Python: Ensure no cross-talk in global tests
By giving all variables unique names

I also added a comment with the function name from the normal tests, so
its' easily visible what these tests are testing
2022-02-25 13:41:51 +01:00
Rasmus Wriedt Larsen
0642610ee9 Python: Global flow works when in own file???
This is very suspicious
2022-02-25 13:36:00 +01:00
Rasmus Wriedt Larsen
d83a9ef8d3 Python: Fix global field-flow for validTest.py 2022-02-25 13:35:43 +01:00
Rasmus Wriedt Larsen
abe4d8da62 Python: Accept global field-flow inconsistencies
Yikes
2022-02-24 15:07:18 +01:00