Commit Graph

206 Commits

Author SHA1 Message Date
Anders Schack-Mulligen
f473a0a961 Python: Deprecate and replace BarrierGuard class. 2022-06-20 15:46:38 +02:00
Anders Schack-Mulligen
6518a01ded Dataflow: Sync. 2022-06-16 11:25:28 +02:00
Tom Hvitved
bcdef98392 Data flow: Sync files 2022-05-25 14:39:37 +02:00
Tom Hvitved
728ccafe2b Merge pull request #9024 from hvitved/dataflow/content-flow-lib
Data flow: Introduce `ContentDataFlow.qll`
2022-05-24 15:09:16 +02:00
Tom Hvitved
f83deb6571 Data flow: Sync files 2022-05-19 15:20:43 +02:00
Tom Hvitved
2b2ac06128 Data flow: Sync files 2022-05-19 13:28:56 +02:00
Nick Rolfe
c518150b49 Merge pull request #9132 from github/nickrolfe/misspelling
QL for QL: generalise non-US spelling query
2022-05-16 16:03:36 +01:00
Anders Schack-Mulligen
83f817ca45 Merge pull request #9134 from aschackmull/dataflow/perf-std-order
Dataflow: Improve standard order through easier type check elimination.
2022-05-16 10:05:17 +02:00
Nick Rolfe
1115227f9d Merge remote-tracking branch 'origin/main' into nickrolfe/misspelling 2022-05-12 16:10:27 +01:00
Nick Rolfe
2efa38aaa6 Python: fix typos in comments 2022-05-12 16:02:20 +01:00
Anders Schack-Mulligen
8c8440a58a Merge pull request #9101 from hvitved/dataflow/include-hidden
Data flow: Add `Configuration::includeHiddenNodes()`
2022-05-12 15:36:12 +02:00
Anders Schack-Mulligen
adb56dfa39 Dataflow: Improve standard order through easier type check elimination. 2022-05-12 14:31:38 +02:00
Anders Schack-Mulligen
4884520ee1 Dataflow: Review fix. 2022-05-11 15:40:49 +02:00
Tom Hvitved
712fe002b9 Data flow: Sync files 2022-05-10 12:41:10 +02:00
Anders Schack-Mulligen
f85e06c2e4 Dataflow: Sync. 2022-05-10 10:12:39 +02:00
Tom Hvitved
d9d5372f28 Data flow: Sync files 2022-05-05 13:36:26 +02:00
yoff
0c7184952b Merge pull request #9023 from RasmusWL/positional-docs
Python: Clarify `getArg` is about positional arguments
2022-05-05 11:28:17 +02:00
Tom Hvitved
66a9759329 Merge pull request #8870 from hvitved/dataflow/expect-content
Data flow: Introduce `expectsContent`
2022-05-05 09:01:40 +02:00
Tom Hvitved
9cb63c0a5e Data flow: Sync files 2022-05-04 14:49:26 +02:00
Tom Hvitved
74e99302d6 Address review comments 2022-05-04 09:57:59 +02:00
Tom Hvitved
da72ba46d4 Data flow: Add stub expectsContent for all languages 2022-05-04 09:57:59 +02:00
Tom Hvitved
6e2e8440eb Data flow: Sync files 2022-05-04 09:57:59 +02:00
Rasmus Wriedt Larsen
d012eaa892 Python: Clarify getArg is about positional arguments 2022-05-03 14:26:23 +02:00
Tom Hvitved
e9c8f979f9 Data flow: Sync files 2022-05-03 11:46:51 +02:00
yoff
1d44694280 Merge pull request #8732 from RasmusWL/dataflow-imports
Python: Don't re-export `python` under `DataFlow::`
2022-05-02 12:08:28 +02:00
Taus
231def026f Merge pull request #8890 from tausbn/python-add-global-attribute-writes
Python: Add support for global attribute writes
2022-05-02 12:03:41 +02:00
Taus
d28f9f41e8 Python: Fix bad join in import_star_read
Makes this

```
(21s) Tuple counts for DataFlowPublic::import_star_read#ff/2@fcd5e6nr after 8.5s:
9743      ~6%     {3} r1 = SCAN num#DataFlowPublic::TModuleVariableNode#fff OUTPUT In.1, In.0, In.2 'result'
9743      ~1%     {3} r2 = JOIN r1 WITH Variables::Variable::getId_dispred#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2 'result'
390808917 ~3%     {3} r3 = JOIN r2 WITH Flow::NameNode::getId_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2 'result'
307       ~0%     {2} r4 = JOIN r3 WITH ImportStar::ImportStar::importStarResolvesTo#ff ON FIRST 2 OUTPUT Lhs.0, Lhs.2 'result'
307       ~0%     {2} r5 = JOIN r4 WITH num#DataFlowPublic::TCfgNode#ff ON FIRST 1 OUTPUT Rhs.1 'n', Lhs.1 'result'
                  return r5
```

become this

```
(17s) Tuple counts for DataFlowPublic::resolved_import_star_module#fff/3@f5e84aic after 0ms:
307 ~0%     {3} r1 = JOIN ImportStar::ImportStar::importStarResolvesTo#ff WITH num#DataFlowPublic::TCfgNode#ff ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'm', Rhs.1 'n'
307 ~0%     {3} r2 = JOIN r1 WITH Flow::NameNode::getId_dispred#ff ON FIRST 1 OUTPUT Lhs.1 'm', Rhs.1 'name', Lhs.2 'n'
            return r2
(17s) Registering DataFlowPublic::resolved_import_star_module#fff/3@f5e84aic +  with content f29281ig38r98icro4ege09mrva
(17s)  >>> Created relation DataFlowPublic::resolved_import_star_module#fff/3@f5e84aic with 307 rows.
(17s) Starting to evaluate predicate DataFlowPublic::import_star_read#ff/2@57b0c06e
(17s) Tuple counts for DataFlowPublic::import_star_read#ff/2@57b0c06e after 2ms:
9743 ~0%     {3} r1 = SCAN num#DataFlowPublic::TModuleVariableNode#fff OUTPUT In.1, In.0, In.2 'result'
9743 ~0%     {3} r2 = JOIN r1 WITH Variables::Variable::getId_dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1, Lhs.2 'result'
307  ~0%     {2} r3 = JOIN r2 WITH DataFlowPublic::resolved_import_star_module#fff ON FIRST 2 OUTPUT Rhs.2 'n', Lhs.2 'result'
             return r3
```
2022-04-28 11:11:37 +00:00
Taus
b4a31e572f Python: Add global attribute writes 2022-04-27 16:45:00 +00:00
Erik Krogh Kristensen
d389012b75 Merge branch 'main' into redundantImport 2022-04-26 14:24:51 +02:00
Anders Schack-Mulligen
c06efa1f42 Dataflow: Sync. 2022-04-25 13:11:04 +02:00
Tom Hvitved
2466288656 Data flow: Simplify revFlowStore 2022-04-25 10:11:54 +02:00
Tom Hvitved
bc6ee10583 Data flow: Sync files 2022-04-22 15:10:00 +02:00
Tom Hvitved
b033f107df Merge remote-tracking branch 'upstream/main' into dataflow/interpret-read-store 2022-04-22 14:35:02 +02:00
Erik Krogh Kristensen
ff73dbc35c delete redundant imports 2022-04-22 12:55:28 +02:00
Anders Schack-Mulligen
677c436e99 Merge pull request #8703 from aschackmull/dataflow/revert-state-in-out-barriers
Dataflow: Revert support for flow-state based in-/out-barriers
2022-04-20 14:54:02 +02:00
Rasmus Wriedt Larsen
084c8eb22e Python: Don't re-export python under DataFlow:: 2022-04-20 11:42:10 +02:00
yoff
0c7130602a Merge pull request #8731 from RasmusWL/delete-old-readme
Python: Delete old dataflow readme
2022-04-20 10:36:12 +02:00
yoff
a66153d73e Merge pull request #8733 from RasmusWL/split-dataflow-private
Python: Split `DataFlowPrivate`
2022-04-20 10:21:05 +02:00
Anders Schack-Mulligen
b521d64156 Dataflow: Sync. 2022-04-19 15:29:35 +02:00
Mathias Vorreiter Pedersen
91b413d59f Dataflow: Sync identical files. 2022-04-19 09:57:21 +01:00
Rasmus Wriedt Larsen
a271e17f04 Python: Move dataflow call-graph to new qll file
Seems like all other languages use a file called `DataFlowDispatch`. I
want to introduce a setup where we have (old) points-to based approach
in one file, and can develop a type-tracking based approach in another
file, so that's the reason for the naming differing slightly.

For which predicates go in which files, I have taken mostly inspiration
from C# and Ruby.
2022-04-13 15:56:57 +02:00
Rasmus Wriedt Larsen
3d15205084 Python: Autoformat 2022-04-13 15:36:16 +02:00
Rasmus Wriedt Larsen
ded4e9250c Python: Move IterableUnpacking to own file 2022-04-13 15:36:05 +02:00
Rasmus Wriedt Larsen
c740894408 Python: Move MatchUnpacking to own file
I had hoped that git would be able to see this as a rename, and
therefore I haven't done autoformat
2022-04-13 15:36:05 +02:00
Rasmus Wriedt Larsen
2e60172bfa Python: Delete old dataflow readme 2022-04-13 12:09:38 +02:00
Anders Schack-Mulligen
7beed570f2 Dataflow: Sync. 2022-04-07 13:53:48 +02:00
Tom Hvitved
4099d1318f Data flow: Tweak two join-orders
Before
```
[2022-04-06 13:19:29] (96s) Tuple counts for DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff/2@i14#aa10f2wi after 4.4s:
                      10681    ~0%     {2} r1 = SCAN DataFlowImpl2::Stage1::revFlow#7ad53399#fff#prev_delta OUTPUT In.0, In.2 'config'
                      982      ~1%     {3} r2 = JOIN r1 WITH DataFlowImpl2::readSet#7ad53399#ffff_2301#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.1 'config', Rhs.2
                      83691528 ~2%     {3} r3 = JOIN r2 WITH DataFlowPublic::ContentSet::getAReadContent#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'config', Lhs.2, Rhs.1 'c'
                      83581763 ~2%     {3} r4 = r3 AND NOT DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff#prev(Lhs.2 'c', Lhs.0 'config')
                      83581763 ~0%     {3} r5 = SCAN r4 OUTPUT In.2 'c', In.0 'config', In.1
                      0        ~0%     {3} r6 = JOIN r5 WITH DataFlowImpl2::Stage1::fwdFlowConsCand#7ad53399#ff ON FIRST 2 OUTPUT Lhs.2, Lhs.1 'config', Lhs.0 'c'
                      0        ~0%     {2} r7 = JOIN r6 WITH DataFlowImpl2::Stage1::fwdFlow#7ad53399#2#fff_02#join_rhs ON FIRST 2 OUTPUT Lhs.2 'c', Lhs.1 'config'
                                       return r7
```

After
```
[2022-04-06 13:44:38] (6s) Tuple counts for DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff/2@i14#5abbf2wn after 6ms:
                      10681  ~0%     {2} r1 = SCAN DataFlowImpl2::Stage1::revFlow#7ad53399#fff#prev_delta OUTPUT In.0, In.2 'config'
                      982    ~1%     {3} r2 = JOIN r1 WITH DataFlowImpl2::readSet#7ad53399#ffff_2301#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.1 'config', Rhs.2
                      109765 ~0%     {3} r3 = JOIN r2 WITH DataFlowImpl2::Stage1::fwdFlowConsCandSet#7ad53399#fff#reorder_0_2_1 ON FIRST 2 OUTPUT Lhs.1 'config', Lhs.2, Rhs.2 'c'
                      0      ~0%     {3} r4 = r3 AND NOT DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff#prev(Lhs.2 'c', Lhs.0 'config')
                      0      ~0%     {3} r5 = SCAN r4 OUTPUT In.1, In.0 'config', In.2 'c'
                      0      ~0%     {2} r6 = JOIN r5 WITH DataFlowImpl2::Stage1::fwdFlow#7ad53399#2#fff_02#join_rhs ON FIRST 2 OUTPUT Lhs.2 'c', Lhs.1 'config'
                                     return r6
```
2022-04-06 13:52:30 +02:00
Tom Hvitved
57f2a74636 Python: Implement ContentSet 2022-04-04 13:51:44 +02:00
Tom Hvitved
c4fbc618a9 Data flow: Sync files 2022-04-04 13:51:44 +02:00
Chris Smowton
3119885a9b Merge pull request #8638 from smowton/smowton/docs/additional-flow-step-description
Improve wording of isAdditionalFlow/TaintStep qldoc
2022-04-01 16:41:04 +01:00