Commit Graph

3152 Commits

Author SHA1 Message Date
Anders Schack-Mulligen
74ae2e0857 Merge pull request #5773 from hvitved/dataflow/aggressive-caching
Data flow: Cache most language-dependent predicates
2021-05-12 09:41:55 +02:00
Tom Hvitved
d66506b0a3 Data flow: Rename {Argument,Parameter}NodeExt to {Arg,Param}Node 2021-05-11 14:40:10 +02:00
yoff
0e5a2c4573 Merge pull request #5442 from jorgectf/jorgectf/python/redos
Python: Add Regular Expression Injection query
2021-05-11 12:11:35 +02:00
yoff
549c9eee1a Merge pull request #5739 from RasmusWL/share-sensitive-data-modeling
Python/JS: Share sensitive data modeling
2021-05-11 11:53:59 +02:00
Rasmus Wriedt Larsen
c2a6b811fc Python: Add modeling of ujson PyPI package
The problem with `tainted_filelike` not having taint, is that in the call

`ujson.dump(tainted_obj, tainted_filelike)`

there is no PostUpdateNote for `tainted_filelike` :( The reason is that
points-to is not able to resolve the call, so none of the clauses in
`argumentPreUpdateNode` matches

See 08731fc6cf/python/ql/src/semmle/python/dataflow/new/internal/DataFlowPrivate.qll (L101-L111)

Let's deal with that issue in an other PR though
2021-05-10 15:10:31 +02:00
Rasmus Wriedt Larsen
72d08f4d6e Python: Model json load/dump 2021-05-10 15:10:30 +02:00
Rasmus Wriedt Larsen
63f28d7d9b Python: Model keyword args to json loads/dumps 2021-05-10 15:10:29 +02:00
Rasmus Wriedt Larsen
3fe9a3d933 Python: Add modeling of simplejson PyPI package
I noticed that we don't handle PostUpdateNote very well in the concept tests,
for exmaple for `json.dump(...)` there _should_ have been an `encodeOutput` as
part of the inline expectations.

I'll work on fixing that up in a separate PR, to keep things clean.
2021-05-10 15:10:27 +02:00
yoff
78370cf63f Update python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll 2021-05-10 14:53:40 +02:00
Rasmus Wriedt Larsen
8afdf26540 Python: Add modeling of idna PyPI package 2021-05-10 11:47:11 +02:00
Rasmus Wriedt Larsen
7ed20a8b2c Python: Add reminder to update docs for new frameworks 2021-05-10 10:55:21 +02:00
thank_you
0238e51c10 Add checks for EmbeddedDocument classes
Mongoengine supports EmbeddedDocument documents. We should check for this in our query.
2021-05-09 19:42:40 -04:00
jorgectf
8665747316 Update sink and sanitizer to match new naming 2021-05-08 18:08:50 +02:00
${sleep,5}
67bc576e30 Delete StdLib.qll 2021-05-07 17:37:02 -04:00
jorgectf
0fc044dfd5 Checkout Stdlib.qll 2021-05-07 23:03:23 +02:00
jorgectf
2ad72ad693 Add LDAP framework entry in Frameworks.qll 2021-05-07 22:16:12 +02:00
jorgectf
6159fbea2b Update functions naming 2021-05-07 22:15:51 +02:00
jorgectf
34b8af30ac Move structure to LDAP.qll 2021-05-07 22:09:57 +02:00
Jorge
c2b96b3a5e Add documentation to main classes' functions.
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-05-07 21:51:10 +02:00
thank_you
aa24c689bc Add back accidentally deleted StdLib.qll file 2021-05-07 15:17:01 -04:00
thank_you
83f0870231 Update file path of module 2021-05-07 15:13:56 -04:00
thank_you
9a44020af3 Rename StdLib.qll file to NoSQL.qll file
It makes more sense to have this file represent just the NoSQL module
2021-05-07 15:13:30 -04:00
thank_you
8f8eff231a Fix comment description of predicate 2021-05-07 15:08:48 -04:00
Jorge
ae806cd445 Merge branch 'github:main' into jorgectf/python/ldapimproperauth 2021-05-07 20:46:09 +02:00
thank_you
1d36aa6649 Add additional querying for mongoengine Document subclassing
After further research, it was discovered that Flask-Mongoengine has multiple ways of allowing a developer to call the Document class. One way is by directly importing the Document class from the module. Another approach is to get the Document class via a mongoengine instance.

The update to this query checks for cases where the developer gets the Document class via the MongoEngine instance.

Other misc changes include setting the various predicates to private.
2021-05-07 14:30:50 -04:00
Rasmus Wriedt Larsen
668bfd3a41 Python: Support EC keygen without class-instance for cryptography
I also added a new test to show off how what the origin ends up looking
like... I think it looks ok
2021-05-05 12:29:55 +02:00
Rasmus Wriedt Larsen
dc4a0c1d38 Python/JS: Fix typo 2021-05-05 10:13:54 +02:00
thank_you
c4a67e522c Rewrite query to take into account MongoClient and subscript expressions
A couple of notes with these changes:

- Added TypeTracker pattern to handle subscript expressions. We've found that pymongo supports subscripts expressions when calling databases and collections. To resolve this, we implemented the TypeTracker pattern to catch those subscripts since CodeQL Python API modeling doesn't support subscript expressions.

- After some research, we've discovered that MongoEngine and Flask-MongoEngine utilize MongoClient under-the-hood. This requires us to rewrite the query so that instead of querying these libraries with specific queries, we are instead going to query for usages of MongoClient since all of the libraries we are targeting utilizes MongoClient under-the-hood.
2021-05-04 19:29:31 -04:00
thank_you
56dc4d886e Add comment on BsonObjectIdCall 2021-05-04 19:11:59 -04:00
CodeQL CI
95f26aadd3 Merge pull request #5681 from yoff/python-support-pathlib
Approved by tausbn
2021-05-04 09:20:24 -07:00
Taus
483199878d Merge pull request #5793 from RasmusWL/fix-qldoc
Python: Minor fix to Django RawSQL QLDoc
2021-05-03 18:18:02 +02:00
intrigus
08731fc6cf Fix typo. 2021-04-29 20:26:34 +02:00
Jorge
bd4b189373 Polish documentation consistency
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-04-29 16:26:28 +02:00
Arthur Baars
6693c5bdd0 Merge pull request #5395 from tausbn/python-share-typetracker
Python: Make the type tracking implementation shareable
2021-04-29 12:06:12 +02:00
CodeQL CI
84d43946de Merge pull request #5755 from RasmusWL/non-alert-data-part1
Approved by tausbn
2021-04-29 02:51:34 -07:00
jorgectf
213d011a8c Edit code example in CompiledRegex
Signed-off-by: jorgectf <jorgectf@protonmail.com>
2021-04-29 11:10:03 +02:00
thank_you
d85b1a2d5f Replace recursive getAMember*() method 2021-04-28 16:54:49 -04:00
Rasmus Lerchedahl Petersen
16bde2729d Python: add flow from methods to calls 2021-04-28 17:02:24 +02:00
Tom Hvitved
c35a2b959a Python: Update data-flow caching 2021-04-28 14:49:05 +02:00
Rasmus Wriedt Larsen
baa926359e Python: Minor fix to Django RawSQL QLDoc 2021-04-28 12:18:27 +02:00
Rasmus Wriedt Larsen
8b9c5f8228 Python/JS: Remove "Only added to aid with internal rewrite" 2021-04-28 11:50:06 +02:00
Taus
4ae3a23089 Python: Limit absolute imports
Limits the behaviour of github/codeql#5614 in two ways:

First, we only consider files that are contained in the source archive.
This prevents unnecessary computation involving files in e.g. the
standard library.

Secondly, we ignore any relative imports (e.g. `from .foo import ...`),
as these only work inside packages anyway.

This fixes an observed performance regression on projects that include
`google-cloud-sdk` as part of their source code.
2021-04-27 21:47:38 +00:00
jorgectf
21e01b809f Add code example in CompiledRegex
Signed-off-by: jorgectf <jorgectf@protonmail.com>
2021-04-27 19:54:42 +02:00
jorgectf
8a800986a2 Remove unused class variables
Signed-off-by: jorgectf <jorgectf@protonmail.com>
2021-04-27 19:54:42 +02:00
jorgectf
20b532ec5e Update to-cast sink's naming
Signed-off-by: jorgectf <jorgectf@protonmail.com>
2021-04-27 19:54:41 +02:00
Jorge
c0c71c509c Apply suggestions from code review
Update `RegexExecution` docs and use `flowsTo()` instead of `getALocalSource()`.

Co-authored-by: yoff <lerchedahl@gmail.com>
2021-04-27 19:54:41 +02:00
jorgectf
c4322848ec Polish qhelp 2021-04-27 19:54:40 +02:00
jorgectf
3fae3fd93e Take ApiGraphs out of Concepts.qll 2021-04-27 19:54:39 +02:00
jorgectf
6a20a4dcc3 Add newline to qhelp 2021-04-27 19:54:38 +02:00
jorgectf
d968eea914 Move expected to /test 2021-04-27 19:54:38 +02:00