Python: Add example of top-level module shadowing stdlib

Although this test is added under the `wrong` folder, the current results from
this CodeQL test is actually correct (compared with the Python
interpreter). However, they don't match what the extractor does when invoked
with `codeql database create`.

Since I deemed it "more than an easy fix" to change the extractor behavior for
`codeql database create` to match the real python behavior, and it turned out to
be quite a challenge to change the extractor behavior for all tests, I'm just
going to make THIS ONE test-case behave like the extractor will with `codeql
database create`...

This is a first commit, to show how the extractor works with qltest by default.

Inspired by the debugging in https://github.com/github/codeql/issues/4640
This commit is contained in:
Rasmus Wriedt Larsen
2020-11-11 15:55:25 +01:00
parent 2945eada9e
commit 8ffcff0824
13 changed files with 67 additions and 0 deletions

View File

@@ -0,0 +1,3 @@
| Local module | cmd.py:0:0:0:0 | Module cmd | referenced in external file called | pdb.py |
| Local module | cmd.py:0:0:0:0 | Module cmd | referenced in local file called | test_ok.py |
| Local module | unique_name.py:0:0:0:0 | Module unique_name | referenced in local file called | unique_name_use.py |

View File

@@ -0,0 +1,13 @@
import python
from ModuleValue mv, ControlFlowNode ref, string local_external
where
ref = mv.getAReference() and
exists(mv.getScope().getFile().getRelativePath()) and
(
if exists(ref.getLocation().getFile().getRelativePath())
then local_external = "local"
else local_external = "external"
)
select "Local module", mv, "referenced in " + local_external + " file called",
ref.getLocation().getFile().getShortName()

View File

@@ -0,0 +1,5 @@
| cmd.py:0:0:0:0 | Module cmd |
| test_fail.py:0:0:0:0 | Module test_fail |
| test_ok.py:0:0:0:0 | Module test_ok |
| unique_name.py:0:0:0:0 | Module unique_name |
| unique_name_use.py:0:0:0:0 | Module unique_name_use |

View File

@@ -0,0 +1,5 @@
import python
from Module m
where exists(m.getFile().getRelativePath())
select m

View File

@@ -0,0 +1,3 @@
| Module 'cmd' (local, not in stdlib, not missing) referenced in local file | test_ok.py:1 |
| Module 'pdb' (external, in stdlib, not missing) referenced in local file | test_fail.py:3 |
| Module 'unique_name' (local, not in stdlib, not missing) referenced in local file | unique_name_use.py:1 |

View File

@@ -0,0 +1,19 @@
import python
from ModuleValue mv, ControlFlowNode ref, string in_stdlib, string local_external, string is_missing
where
ref = mv.getAReference() and
exists(ref.getLocation().getFile().getRelativePath()) and
(
if mv.getScope().getFile().inStdlib()
then in_stdlib = "in stdlib"
else in_stdlib = "not in stdlib"
) and
(
if exists(mv.getScope().getFile().getRelativePath())
then local_external = "local"
else local_external = "external"
) and
(if mv.isAbsent() then is_missing = "missing" else is_missing = "not missing")
select "Module '" + mv.getName() + "' (" + local_external + ", " + in_stdlib + ", " + is_missing +
") referenced in local file", ref.getLocation().toString()

View File

@@ -0,0 +1,8 @@
This test shows how we handle modules the shadow a module in the standard library.
We manually replicate the behavior of `codeql database create --source-root <src-dir>`, which will use `-R <src-dir>`. By default, the way qltest invokes the extractor will cause different behavior. Therefore, we also need to move our code outside of the top-level folder, and it lives in `code/`.
Because we have a `cmd.py` file, whenever the python interpreter sees `import cmd`, that is the file that will be used! --
* `python test_ok.py` works as intended, and prints `Foo`
* `python test_fail.py` raises an exception, since it imports `pdb.py` from the standard library, which (at least in Python 3.8) tries to import `cmd.py` from the standard library, but instead is served our `cmd.py` module. Therefore it fails with `AttributeError: module 'cmd' has no attribute 'Cmd'`

View File

@@ -0,0 +1,2 @@
foo = "Foo"
print("my own cmd imported")

View File

@@ -0,0 +1 @@
semmle-extractor-options: --max-import-depth=2

View File

@@ -0,0 +1,3 @@
# we import `pdb` which import the `cmd` module from the standard library
# and allows us to set --max-import-depth=2, to make the test run fast
import pdb

View File

@@ -0,0 +1,2 @@
from cmd import foo
print(foo)

View File

@@ -0,0 +1,2 @@
from unique_name import foo
print(foo)