Depends on an internal PR. The two lines in question were caused by
the insertion of an extra node due to the failure to parse a trailing
comma corrcetly.
Since `python.qll` has `private import
semmle.python.dataflow.new.DataFlow`, that means that all tests now
implicitly imports the frameworks modeling, and therefore any python
class is part of the DjangoViewClassHelper ql class.
de8ecb214f/python/ql/lib/python.qll (L44)
This reverts commit 8d15680af4, reversing
changes made to 63831cc62b.
This PR caused performance problems, so reverting now to clear up immediate
problems.
Fixes the import logic to account for absolute imports.
We do this by classifying which files and folders may serve as the
entry point for execution, based on a few simple heuristics. If the
file `module.py` is in the same folder as a file `main.py` that may be
executed directly, then we allow `module` to be a valid name for
`module.py` so that `import module` will work as expected.
This one will require some explanation...
First, the file structure. This commit adds a test consisting
representing a few different kinds of imports.
- Absolute imports, from `module.py` to `main.py` when the latter is
executed directly.
- A package (contained in the `package` folder)
- A namespace package (contained in the `namespace_package` folder)
All of these are inside a folder called `code` for reasons I will
detail later.
The file `main.py` is identified as a script, by the presence of the
`!#` comment in its first line.
The files themselves are executable, and `python3 main.py` will print
out all modules in the order they are imported.
The test itself is very simple. It simply lists all modules and their
corresponding names. As is plainly visible, without modification we
only pick up `package` and its component modules as having names. This
is the bit that needs to be fixed.
Convincing the test runner to extract this test in a way that mimics
reality is, unfortunately, a bit complicated. By default, the test
runner itself includes any Python files in the test directory as
modules in the invocation of the extractor, and so we must hide
everything in the `code` subdirectory.
Secondly, a `--path` argument (set to the test directory) is
automatically added, and this would also interfere with extraction,
and hence we must prevent this. Luckily, if we supply our own `--path`
argument -- even if it doesn't make any sense -- then the other
argument is left out.
Finally, we must actually tell the extractor to extract the files (or
it would just happily pass the test with zero files extracted), so the
`-R .` argument ensures that we recurse over the files in the test
directory after all.
Default values for positional arugments follow a rule, so if an argument has a
default value, later positional arguments must also have default values.
The database only stores the actual default values, and nothing about the
arguments that doesn't have default values.
This turns out to be a major problem for Argument.getKwDefault(i), since default
values for keyword-only arguments doesn't have the same rule. So if you know
there is one default value, you can't tell if it is associated with `foo` or
`bar`, as in the examples below:
```
def a(*, foo=None, bar):
pass
def b(*, foo, bar=None):
pass
```
How could the tests fail because of autoformatting, you may ask?
The answer is deprecation warnings. These specify the location of the deprecated
entity, and due to autoformatting these moved around.
This is a temporary fix!
Added minimal working example (MWE) as a regression, so it's easier to fix the
real problem.
only Python 3 is facing the problem -- and without --max-import-depth=1 the test
times out at 10 minutes :O
We can't understand the real `six.py` file, so we have some internal plumbing
that enables us to handle six anyway. While updating that, I had a hell of a lot
of trouble with these tests.
What we actually want, is to see that we can understand what the values imported
from six are (i.e., their points-to information). I added a few more, that I
think would be useful. If we can figure out all of these, I don't actually care
if we're doing it by understanding the real `six.py` file, or by some internal
trick.
I verified that we don't get results with the real `six.py` file by disabling
our internal tricks, and putting a copy of six.py just next to test.py.
We used to have an other file that would list all the properties we knew and
their value, but that turned out to be a fragile and annoying test, since the
results differed from which version of python you ran it with (3.5 vs 3.8) and
which machine you ran it on (my machien vs jenkins). I don't care about the
results in this file, and I can certainly not eyeball it to see if it's correct
or not.
