Commit Graph

2826 Commits

Author SHA1 Message Date
github-actions[bot]
2b939fdf08 Post-release preparation for codeql-cli-2.15.2 2023-10-30 16:06:51 +00:00
Harry Maclean
083be305e1 Shared: Add neutralModel extensible predicate
The neutralModel extensible predicate already exists in Java and C#, so
this change brings the dynamic languages more in line with static
languages. The Model Editor uses this predicate to mark endpoints as
"not interesting" from a data flow perspective.
2023-10-30 11:31:57 +00:00
github-actions[bot]
4641990021 Release preparation for version 2.15.2 2023-10-30 11:05:53 +00:00
Tom Hvitved
bf3fb685ad Python: Sync in changes to SummaryTypeTracker.qll 2023-10-30 10:52:07 +01:00
Rasmus Lerchedahl Petersen
1f8a51cffe Python: add module for post-update flow
The logic for adding flow out of post-update-nodes
was also duplicated, so I added a module for that also.

The two modules are now only used together. I did not
fuse them, though, as I think they may not be used
together in the case of use-use-flow.
2023-10-27 15:07:49 +02:00
Rasmus Lerchedahl Petersen
e745df6478 Python: module for import time flow
The logic for separating local flow into _import time_
and _runtime_ was duplicated a few times.
Create a module for it instead, and add a good qldoc.
2023-10-27 15:07:49 +02:00
yoff
867a39083e Merge pull request #14114 from yoff/python/allow-namespace-packages
Python: Allow namespace packages
2023-10-26 16:56:05 +02:00
Rasmus Lerchedahl Petersen
dcc778520a Python: refactor code
Also add explanatory comment.

Co-authored-by: Taus <tausbn@github.com>
2023-10-26 15:00:02 +02:00
Rasmus Lerchedahl Petersen
50041f07a3 Python: fix comment 2023-10-26 14:28:00 +02:00
Max Schaefer
3939167ba2 Include more details in the message for py/weak-cryptographic-algorithm.
Specifically, we add a link to the location where the cryptographic algorithm is configured, which can be far away from its use.
2023-10-26 11:28:09 +01:00
Rasmus Wriedt Larsen
be494b73d4 Python: Minor cleanup for string pool interaction
This doesn't really seem to impact performance, is just a bit cleaner way to write the QL code.

In the worst case, since we do string concat with empty string, we would need to load string + attempt to store new string in string pool, which is useless since we can surely just reuse the integer ID.

-- Previously

Evaluated non-recursive predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0#join_rhs#CSE@5fdc6eia in 1ms (size: 2).
Evaluated relational algebra for predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0#join_rhs#CSE@5fdc6eia with tuple counts:
         1   ~0%    {1} r1 = CONSTANT(unique string)[""]

         1   ~0%    {1} r2 = CONSTANT(unique string)[".__init__"]

         2   ~0%    {1} r3 = r1 UNION r2
                    return r3

Evaluated recursive predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0@b4f54xlv in 72ms on iteration 1 (delta size: 26339).
Evaluated relational algebra for predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0@b4f54xlv on iteration 1 running pipeline base with tuple counts:
         6382   ~0%    {3} r1 = JOIN ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0#join_rhs#CSE WITH Module#68efc075::Module::getFile#0#dispred#ff CARTESIAN PRODUCT OUTPUT Rhs.1, Lhs.0, Rhs.0
         6066   ~2%    {4} r2 = JOIN r1 WITH Module#68efc075::isPreferredModuleForName#2#ff ON FIRST 1 OUTPUT Lhs.2, _, Lhs.1, Rhs.1
                       {2} r3 = REWRITE r2 WITH Tmp.1 := "", Out.1 := InverseAppend(Tmp.1,In.2,In.3) KEEPING 2
         3721   ~0%    {2} r4 = SCAN r3 OUTPUT In.1, In.0
        25945   ~2%    {2} r5 = JOIN r4 WITH ImportResolution#f829f0f0::ImportResolution::getReferenceToModuleName#1#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1

           38   ~4%    {3} r6 = SCAN SsaDefinitions#9197156e::SsaSource::init_module_submodule_defn#2#ff OUTPUT In.1, _, In.0
           38   ~1%    {3} r7 = REWRITE r6 WITH Out.1 := -1
           38   ~1%    {2} r8 = JOIN r7 WITH py_scope_flow_021#join_rhs ON FIRST 2 OUTPUT Rhs.2, Lhs.2
           38   ~0%    {2} r9 = JOIN r8 WITH Module#68efc075::Module::getPackageName#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
          394   ~2%    {2} r10 = JOIN r9 WITH Essa#24e22a14::EssaVariable::getSourceVariable#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
          394   ~0%    {3} r11 = JOIN r10 WITH DataFlowPublic#c4b6ce7e::EssaNode#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Rhs.1

          394   ~0%    {5} r12 = JOIN r11 WITH Essa#24e22a14::EssaVariable::getName#0#dispred#ff ON FIRST 1 OUTPUT _, Lhs.2, Lhs.1, Rhs.1, _
          394   ~0%    {2} r13 = REWRITE r12 WITH Tmp.0 := ".", Tmp.4 := "", Out.0 := (In.2 ++ Tmp.0 ++ In.3 ++ Tmp.4) KEEPING 2

          394   ~0%    {5} r14 = JOIN r11 WITH Essa#24e22a14::EssaVariable::getName#0#dispred#ff ON FIRST 1 OUTPUT _, Lhs.2, Lhs.1, Rhs.1, _
          394   ~1%    {2} r15 = REWRITE r14 WITH Tmp.0 := ".", Tmp.4 := ".__init__", Out.0 := (In.2 ++ Tmp.0 ++ In.3 ++ Tmp.4) KEEPING 2

          788   ~3%    {2} r16 = r13 UNION r15
          401   ~2%    {2} r17 = JOIN r16 WITH Module#68efc075::isPreferredModuleForName#2#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
          394   ~1%    {2} r18 = JOIN r17 WITH Module#68efc075::Module::getFile#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Rhs.1

        26339   ~1%    {2} r19 = r5 UNION r18
                       return r19

-- Now

Evaluated recursive predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0@409a0xhe in 8ms on iteration 1 (delta size: 26339).
Evaluated relational algebra for predicate ImportResolution#f829f0f0::ImportResolution::getImmediateModuleReference#1#ff#reorder_1_0@409a0xhe on iteration 1 running pipeline base with tuple counts:
         3033   ~0%    {2} r1 = JOIN Module#68efc075::Module::getFile#0#dispred#ff_10#join_rhs WITH Module#68efc075::isPreferredModuleForName#2#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1

         3033   ~0%    {4} r2 = JOIN Module#68efc075::Module::getFile#0#dispred#ff_10#join_rhs WITH Module#68efc075::isPreferredModuleForName#2#ff ON FIRST 1 OUTPUT Lhs.1, _, Rhs.1, _
                       {2} r3 = REWRITE r2 WITH Tmp.1 := "", Tmp.3 := ".__init__", Out.1 := InverseAppend(Tmp.1,Tmp.3,In.2) KEEPING 2
          688   ~0%    {2} r4 = SCAN r3 OUTPUT In.1, In.0

         3721   ~0%    {2} r5 = r1 UNION r4
        25945   ~2%    {2} r6 = JOIN r5 WITH ImportResolution#f829f0f0::ImportResolution::getReferenceToModuleName#1#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1

           38   ~4%    {3} r7 = SCAN SsaDefinitions#9197156e::SsaSource::init_module_submodule_defn#2#ff OUTPUT In.1, _, In.0
           38   ~1%    {3} r8 = REWRITE r7 WITH Out.1 := -1
           38   ~1%    {2} r9 = JOIN r8 WITH py_scope_flow_021#join_rhs ON FIRST 2 OUTPUT Rhs.2, Lhs.2
           38   ~0%    {2} r10 = JOIN r9 WITH Module#68efc075::Module::getPackageName#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
          394   ~2%    {2} r11 = JOIN r10 WITH Essa#24e22a14::EssaVariable::getSourceVariable#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
          394   ~0%    {3} r12 = JOIN r11 WITH DataFlowPublic#c4b6ce7e::EssaNode#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Rhs.1

          394   ~2%    {4} r13 = JOIN r12 WITH Essa#24e22a14::EssaVariable::getName#0#dispred#ff ON FIRST 1 OUTPUT _, Lhs.2, Lhs.1, Rhs.1
          394   ~0%    {2} r14 = REWRITE r13 WITH Tmp.0 := ".", Out.0 := (In.2 ++ Tmp.0 ++ In.3) KEEPING 2

          394   ~0%    {5} r15 = JOIN r12 WITH Essa#24e22a14::EssaVariable::getName#0#dispred#ff ON FIRST 1 OUTPUT _, Lhs.2, Lhs.1, Rhs.1, _
          394   ~5%    {2} r16 = REWRITE r15 WITH Tmp.0 := ".", Tmp.0 := (In.2 ++ Tmp.0 ++ In.3), Tmp.4 := ".__init__", Out.0 := (Tmp.0 ++ Tmp.4) KEEPING 2

          788   ~0%    {2} r17 = r14 UNION r16
          401   ~1%    {2} r18 = JOIN r17 WITH Module#68efc075::isPreferredModuleForName#2#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
          394   ~1%    {2} r19 = JOIN r18 WITH Module#68efc075::Module::getFile#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Rhs.1

        26339   ~0%    {2} r20 = r6 UNION r19
                       return r20
2023-10-25 14:53:51 +01:00
Rasmus Wriedt Larsen
e8f548ab52 Python: Model routed parameter flow to *args and **kwargs in Django + rest framework 2023-10-23 17:18:22 +02:00
amammad
a0e2e1ef21 add to Frameworks.qll 2023-10-21 19:49:27 +02:00
amammad
1fe565a46f cherrypy framework file system access Sinks are added 2023-10-21 19:47:30 +02:00
Mathew Payne
a24e168ec0 Merge branch 'main' into py-restframework 2023-10-20 11:39:07 +01:00
Dave Bartolomeo
712f7758cf Merge branch 'main' into post-release-prep/codeql-cli-2.15.1 2023-10-19 12:14:07 -04:00
github-actions[bot]
8dcd8b9e5b Post-release preparation for codeql-cli-2.15.1 2023-10-17 20:24:00 +00:00
Rasmus Wriedt Larsen
62a992473f Python: Update QLDoc 2023-10-17 10:09:19 +02:00
Rasmus Wriedt Larsen
2bf4c32433 Python: Add syntactic support for yield in contextlib.contextmanager 2023-10-17 09:51:20 +02:00
github-actions[bot]
3b3c036626 Release preparation for version 2.15.1 2023-10-16 17:49:39 +00:00
Henry Mercer
1a370bfbbe Merge pull request #14443 from github/post-release-prep/codeql-cli-2.15.0
Post-release preparation for codeql-cli-2.15.0
2023-10-11 17:39:04 +01:00
github-actions[bot]
ae6af17c74 Post-release preparation for codeql-cli-2.15.0 2023-10-11 14:19:20 +00:00
Rasmus Wriedt Larsen
68d00a829e Merge pull request #14430 from RasmusWL/api-graph-import-star
Python: Better allow `import *` to work with API graphs
2023-10-11 10:03:46 +02:00
Rasmus Wriedt Larsen
ee75b104eb Python: Add change-note 2023-10-10 17:45:11 +02:00
Rasmus Wriedt Larsen
72d0dcdaba Python: Workaround for module level items from import * not being LocalSourceNodes 2023-10-10 17:45:11 +02:00
yoff
f1266a3e81 Merge pull request #14417 from github/tausbn/python-add-flow-for-assignment-expressions 2023-10-10 17:09:20 +02:00
Rasmus Wriedt Larsen
2d947a4f53 Merge pull request #13781 from maikypedia/maikypedia/python-unsafe-deserialization
Python: Add unsafe deserialization sinks (CWE-502)
2023-10-10 13:30:38 +02:00
Taus
e8ac258994 Python: Add missing flow for AssignmentExpr nodes
Also extend the tests surrounding this construct to be a bit more comprehensive.

Co-authored-by: Rasmus Lerchedahl Petersen <yoff@github.com>
2023-10-09 14:16:03 +00:00
erik-krogh
a7ab9fd93b add change-notes 2023-10-09 09:43:06 +02:00
erik-krogh
194f918c0b Python: delete various outdated deprecations 2023-10-09 09:14:55 +02:00
erik-krogh
1c9f59e491 Python:delete deprecated files modelling web frameworks 2023-10-09 09:14:54 +02:00
erik-krogh
0d992a3d1f delete old deprecated aliases of various regex libraries 2023-10-09 09:14:54 +02:00
amammad
ad2631202d fix comments 2023-10-08 21:32:04 +02:00
amammad
6c8cc79b4d v1 2023-10-08 21:24:54 +02:00
erik-krogh
4bc4e0845d delete the deprecated isBarrierGuard predicate from the shared dataflow library, and its uses 2023-10-07 21:48:49 +02:00
Cornelius Riemenschneider
d3a1dbc0c7 Merge pull request #14381 from github/criemen/add-bazel-dbschemes
Add skeleton bazel files for accessing the dbschemes.
2023-10-05 16:53:45 +02:00
Cornelius Riemenschneider
96edc1d349 Add skeleton bazel files for accessing the dbschemes. 2023-10-05 09:00:38 +02:00
github-actions[bot]
9fe993bec3 Release preparation for version 2.15.0 2023-10-04 14:15:27 +00:00
Rasmus Wriedt Larsen
9c02b4f21c Merge pull request #14289 from microsoft/jb1/16-cryptography-models-libraries-and-queries-migration
16 cryptography models libraries and queries migration
2023-10-04 12:27:59 +02:00
Josh Brown
de2e8b0b12 explicit "this" qualifiers 2023-10-03 16:13:54 -07:00
Josh Brown
ad86e576a4 autoformat 2023-10-03 13:40:17 -07:00
Josh Brown
b683a3caf8 Merge branch 'main' into jb1/16-cryptography-models-libraries-and-queries-migration 2023-10-04 07:24:29 +11:00
Henry Mercer
da92da2204 Bump minor versions of packs we regularly release 2023-10-03 16:31:23 +01:00
Henry Mercer
f3847b3f51 Merge branch 'main' into henrymercer/rc-3.11-mergeback 2023-10-03 16:30:23 +01:00
Mathew Payne
3ab5fd5ca4 Add RestFramework handler kwargs 2023-10-02 14:58:21 +01:00
Mathew Payne
41bb8377d9 Add change notes 2023-09-29 14:44:36 +01:00
Rasmus Lerchedahl Petersen
177db998c7 Python: add change note 2023-09-29 15:28:08 +02:00
Mathew Payne
eb9b32473e Add support for ModelViewSet functions 2023-09-29 14:26:39 +01:00
Rasmus Lerchedahl Petersen
ed3ffde5e6 Python: modules are now possibly non-unique
We should consider if this is the right way..
2023-09-29 15:10:19 +02:00
Rasmus Lerchedahl Petersen
4f35a62583 Python: broaden search for imports
This now finds vulnerabilities in
https://github.com/github/field-security-codeql/issues/100
2023-09-29 15:10:19 +02:00