Commit Graph

3072 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
74ca1d00b9 Python: More precise regex parsing 2021-06-28 17:04:48 +02:00
Rasmus Lerchedahl Petersen
e5f07cc4d3 Python: inline test of regex components
- Added naive implementation of `charRange` so the test can run.
- Made predicates public as needed.
2021-06-28 17:04:48 +02:00
jorgectf
1d4d8ab6e0 Fix tests 2021-06-28 14:16:52 +02:00
jorgectf
b9422518b3 Rephrase .qhelp 2021-06-28 14:00:00 +02:00
Rasmus Wriedt Larsen
318694ccc8 Python: Don't rely on d = d.getOutput() for Decoding
Although it is for `json.loads` and the like.
2021-06-28 13:17:45 +02:00
Rasmus Wriedt Larsen
59711424bd Python: Fix qhelp for NoSQL injection 2021-06-28 11:48:28 +02:00
Rasmus Wriedt Larsen
5477b2e0d5 Python: Minor refactoring cleanup 2021-06-28 10:54:21 +02:00
Rasmus Wriedt Larsen
4a2c99a021 Python: Inline LDAPImproperAuth.qll
Since having it inlined makes the query a bit easier to read. We
obviously need to share it if we want to share this predicate, but for
now that does not seem to be the case.
2021-06-28 10:54:21 +02:00
Rasmus Wriedt Larsen
b33f6a315c Python: Fix select for py/improper-ldap-auth 2021-06-28 10:54:21 +02:00
Rasmus Wriedt Larsen
dfe16aae4c Python: Handle both positional and keyword args for LDAP bind 2021-06-28 10:46:13 +02:00
Rasmus Wriedt Larsen
97571e0b4f Python: Add modeling of peewee 2021-06-25 17:50:59 +02:00
Rasmus Wriedt Larsen
1317ae298c Python: Rename cursor => Cursor in PEP249
Notice that since this will be part of the same PR as 5cfc433, it is OK
to do this change without keeping `PEP249::cursor` for backwards
compatibility.
2021-06-25 17:30:35 +02:00
Rasmus Wriedt Larsen
d8db83d081 Python: Add cursor::instance for PEP249
For Peewee modeling I want to be able to define new cursor instances
just like I can do for connections.
2021-06-25 17:29:32 +02:00
Rasmus Wriedt Larsen
6be0db2c22 Python: Improve QLDoc of PEP249 modeling 2021-06-25 17:24:28 +02:00
Rasmus Wriedt Larsen
5cfc43395b Python: Refactor PEP249 to encapsulate in module
So global namespace doesn't contain `Connection` whenever `PEP249.qll`
is imported
2021-06-25 17:15:12 +02:00
Rasmus Wriedt Larsen
9573048ee8 Python: Port py/clear-text-logging-sensitive-data 2021-06-25 14:35:31 +02:00
Rasmus Wriedt Larsen
68cfeb0b5c Python: Model logging from the logging module 2021-06-25 14:26:35 +02:00
Rasmus Wriedt Larsen
c05e375401 Python: Fix indentation of hashlib modeling 2021-06-25 14:26:35 +02:00
Rasmus Wriedt Larsen
36c9ceb13b Python: Add Logging concept 2021-06-25 14:26:35 +02:00
Rasmus Wriedt Larsen
a7eb1b3a12 Python: Minor QLDoc fixup 2021-06-25 14:26:35 +02:00
Anders Schack-Mulligen
2d24387e9e Merge pull request #6149 from edoardopirovano/fix-java-regression
Performance: Fix bad join order in Java dataflow library
2021-06-25 10:42:05 +02:00
Rasmus Wriedt Larsen
a9469b73d9 Python: Port py/clear-text-storage-sensitive-data 2021-06-24 17:39:08 +02:00
Rasmus Wriedt Larsen
7017beca47 Python: Model CookieWrite for twisted
Had to split the call to `request.cookies.append` since inline
expectation tests didn't like the expectation that contained `=` :(
2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
4606444b85 Python: Model CookieWrite for flask 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
65c526df86 Python: Model CookieWrite for tornado 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
9340d658a4 Python: Model CookieWrite for django 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
930ed0a712 Python: Minor django fixup 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
226425e831 Python: Model CookieWrite for aiohttp 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
e1af1f11ee Python: Add HTTP::Server::CookieWrite concept
along with tests, but no implementations (to ease reviewing).

---

I've put quite some thinking into what to call our concept for this.

[JS has `CookieDefinition`](581f4ed757/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll (L148-L187)), but I couldn't find a matching concept in any other languages.

We used to call this [`CookieSet`](f07a7bf8cf/python/ql/src/semmle/python/web/Http.qll (L76)) (and had a corresponding `CookieGet`).

But for headers, [Go calls this `HeaderWrite`](cd1e14ed09/ql/src/semmle/go/concepts/HTTP.qll (L97-L131)) and [JS calls this `HeaderDefinition`](581f4ed757/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll (L23-L46))

I think it would be really cool if we have a naming scheme that means the name for getting the value of a header on a incoming request is obvious. I think `HeaderWrite`/`HeaderRead` fulfils this best. We could go with `HeaderSet`/`HeaderGet`, but they feel a bit too vague to me. For me, I'm so used to talking about def-use, that I would immediately go for `HeaderDefinition` and `HeaderUse`, which could work, but is kinda strange.

So in the end that means I went with `CookieWrite`, since that allows using a consistent naming scheme for the future :)
2021-06-24 17:34:43 +02:00
Anders Schack-Mulligen
95ad8b55fe Merge pull request #6107 from aschackmull/dataflow/implicit-reads
Dataflow: Add support for implicit reads
2021-06-24 15:38:35 +02:00
Anders Schack-Mulligen
cd0efbe7ce Dataflow: Sync. 2021-06-24 14:19:17 +02:00
Rasmus Wriedt Larsen
686638a65f Merge pull request #6049 from RasmusWL/jmespath
Python: Add modeling of `jmespath`
2021-06-24 11:13:19 +02:00
Edoardo Pirovano
0909c9ff22 Performance: Fix bad join order in dataflow library 2021-06-24 08:24:17 +01:00
CodeQL CI
469e709113 Merge pull request #6055 from RasmusWL/rsa-modeling
Approved by yoff
2021-06-23 08:35:25 -07:00
Rasmus Wriedt Larsen
0774e985ce Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-06-23 13:37:38 +02:00
Rasmus Wriedt Larsen
902b450b12 Python: Also model pathlib.Path().open().write()
And this transition to type-trackers also helped fix the missing path
through function calls 👍
2021-06-23 10:50:04 +02:00
Rasmus Wriedt Larsen
39ec8701ca Python: Add FileSystemWriteAccess concept
I made `FileSystemWriteAccess` be a subclass of `FileSystemAccess` (like in [JS](64001cc02c/javascript/ql/src/semmle/javascript/Concepts.qll (L68-L74))), but then I started wondering about how I could  give a good result for `getAPathArgument`, and what would a good result even be? The argument to the `open` call, or the object that the `write` method is called on? I can't see how doing either of these enables us to do anything useful...

So I looked closer at how JS uses `FileSystemWriteAccess`:

1. as sink for zip-slip: 7c51dff0f7/javascript/ql/src/semmle/javascript/security/dataflow/ZipSlipCustomizations.qll (L121)
2. as sink for downloading unsafe files (identified through their extension) through non-secure connections: 89ef6ea4eb/javascript/ql/src/semmle/javascript/security/dataflow/InsecureDownloadCustomizations.qll (L134-L150)
3. as sink for writing untrusted data to a local file  93b1e59d62/javascript/ql/src/semmle/javascript/security/dataflow/HttpToFileAccessCustomizations.qll (L43-L46)

for the 2 first sinks, it's important that `getAPathArgument` has a proper result... so that solves the problem, and highlights that it _can_ be important to give proper results for `getAPathArgument` (if possible).

So I'm trying to do best effort for `f = open(...); f.write(...)`, but with this current code we won't always be able to give a result (as highlighted by the tests). It will also be the case that there are multiple `FileSystemAccess` with the same path-argument, which could be a little strange.

overall, I'm not super confident about the way this new concept and implementation turned out, but it also seems like the best I could come up with right now...

The obvious alternative solution is to NOT make `FileSystemWriteAccess` a subclass of `FileSystemAccess`, but I'm not very tempted to go down this path, given the examples of this being useful above, and just the general notion that we should be able to model writes as being a specialized kind of `FileSystemAccess`.
2021-06-23 10:50:04 +02:00
Rasmus Wriedt Larsen
c0964617d7 Merge pull request #6111 from tausbn/python-a-few-minor-cleanups
Python: A few minor bits of cleanup
2021-06-23 10:42:41 +02:00
Rasmus Wriedt Larsen
c77884b8c4 Python: Apply suggestions from code review
Co-authored-by: Jorge <46056498+jorgectf@users.noreply.github.com>
2021-06-23 10:37:06 +02:00
Rasmus Wriedt Larsen
ae841cdd9b Python: Make import python private in Concepts.qll
Just a mistake that we have never caught on to
2021-06-23 10:32:36 +02:00
Taus
317c6867aa Python: Fix sneaky semantic change
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-06-22 16:46:54 +02:00
Rasmus Wriedt Larsen
3b41c2f204 Python: Use new MethodCallNode in TaintTrackingPrivate 2021-06-22 15:12:35 +02:00
Rasmus Wriedt Larsen
0b767bb853 Merge branch 'main' into small-cleanups 2021-06-22 15:01:53 +02:00
Rasmus Wriedt Larsen
5db627042f Merge pull request #6091 from tausbn/python-exclude-main-py-files
Python: Avoid `__main__.py` files as entry points.
2021-06-22 11:29:02 +02:00
Rasmus Wriedt Larsen
e05d6e71b8 Merge pull request #6064 from tausbn/python-add-get-method-call
Python: Add `getAMethodCall` to `LocalSourceNode`
2021-06-22 11:16:39 +02:00
Taus
ba6ab8ff3d Python: Expand __main__.py comment
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-06-21 18:14:03 +02:00
Taus
768cab3642 Python: Address review comments
- changes `getReceiver` to `getObject`
- fixes `calls` to avoid unwanted cross-talk
- adds some more documentation to highlight the above issue
2021-06-21 14:57:19 +00:00
Rasmus Wriedt Larsen
1c48aca630 Merge branch 'main' into jmespath 2021-06-21 15:26:45 +02:00
CodeQL CI
565af1a879 Merge pull request #6071 from RasmusWL/fix-input-cwe
Approved by calumgrant, tausbn
2021-06-21 06:23:18 -07:00
Rasmus Wriedt Larsen
a7170bedb6 Python: Mention modeling of mysqlclient PyPI package
Just for completeness in terms of what we claim support for.
2021-06-21 15:20:08 +02:00