Manual merge

This commit is contained in:
Josh Brown
2025-07-10 14:25:18 -07:00
parent 5fb45c89e9
commit 4c5945f4aa
1701 changed files with 175943 additions and 32094 deletions

View File

@@ -1,3 +1,13 @@
## 1.6.0
### Query Metadata Changes
* The tag `quality` has been added to multiple Python quality queries for consistency. They have all been given a tag for one of the two top-level categories `reliability` or `maintainability`, and a tag for a sub-category. See [Query file metadata and alert message style guide](https://github.com/github/codeql/blob/main/docs/query-metadata-style-guide.md#quality-query-sub-category-tags) for more information about these categories.
### Minor Analysis Improvements
* The `py/iter-returns-non-self` query has been modernized, and no longer alerts for certain cases where an equivalent iterator is returned.
## 1.5.2
### Minor Analysis Improvements

View File

@@ -2,9 +2,9 @@
* @name Conflicting attributes in base classes
* @description When a class subclasses multiple base classes and more than one base class defines the same attribute, attribute overriding may result in unexpected behavior by instances of this class.
* @kind problem
* @tags reliability
* maintainability
* modularity
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity low
* @precision high

View File

@@ -2,7 +2,8 @@
* @name `__eq__` not overridden when adding attributes
* @description When adding new attributes to instances of a class, equality for that class needs to be defined.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Inconsistent equality and hashing
* @description Defining equality for a class without also defining hashability (or vice-versa) violates the object model.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-581
* @problem.severity warning

View File

@@ -2,7 +2,8 @@
* @name Inconsistent method resolution order
* @description Class definition will raise a type error at runtime due to inconsistent method resolution order(MRO)
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity high

View File

@@ -0,0 +1,48 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
When initializing an instance of the class in the class' <code>__init__</code> method, calls tha are made using the instance may receive an instance of the class that is not
yet fully initialized. When a method called in an initializer is overridden in a subclass, the subclass method receives the instance
in a potentially unexpected state. Fields that would be initialized after the call, including potentially in the subclass' <code>__init__</code> method,
will not be initialized. This may lead to runtime errors, as well as make the code more difficult to maintain, as future changes may not
be aware of which fields would not be initialized.
</p>
</overview>
<recommendation>
<p>If possible, refactor the initializer method such that initialization is complete before calling any overridden methods.
For helper methods used as part of initialization, avoid overriding them, and instead call any additional logic required
in the subclass' <code>__init__</code> method.
</p>
<p>
If the overridden method does not depend on the instance <code>self</code>, and only on its class, consider making it a <code>@classmethod</code> or <code>@staticmethod</code> instead.
</p>
<p>
If calling an overridden method is absolutely required, consider marking it as an internal method (by using an <code>_</code> prefix) to
discourage external users of the library from overriding it and observing partially initialized state, and ensure that the fact it is called during initialization
is mentioned in the documentation.
</p>
</recommendation>
<example>
<p>In the following case, the <code>__init__</code> method of <code>Super</code> calls the <code>set_up</code> method that is overridden by <code>Sub</code>.
This results in <code>Sub.set_up</code> being called with a partially initialized instance of <code>Super</code> which may be unexpected. </p>
<sample src="examples/InitCallsSubclassMethodBad.py" />
<p>In the following case, the initialization methods are separate between the superclass and the subclass.</p>
<sample src="examples/InitCallsSubclassMethodGood.py" />
</example>
<references>
<li>CERT Secure Coding: <a href="https://www.securecoding.cert.org/confluence/display/java/MET05-J.+Ensure+that+constructors+do+not+call+overridable+methods">
Rule MET05-J</a>. Reference discusses Java but is applicable to object oriented programming in many languages.</li>
<li>StackOverflow: <a href="https://stackoverflow.com/questions/3404301/whats-wrong-with-overridable-method-calls-in-constructors">Overridable method calls in constructors</a>.</li>
<li>Python documentation: <a href="https://docs.python.org/3/library/functions.html#classmethod">@classmethod</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,58 @@
/**
* @name `__init__` method calls overridden method
* @description Calling a method from `__init__` that is overridden by a subclass may result in a partially
* initialized instance being observed.
* @kind problem
* @tags reliability
* correctness
* quality
* @problem.severity warning
* @sub-severity low
* @precision high
* @id py/init-calls-subclass
*/
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.internal.DataFlowDispatch
predicate initSelfCallOverridden(
Function init, DataFlow::Node self, DataFlow::MethodCallNode call, Function target,
Function override
) {
init.isInitMethod() and
call.getScope() = init and
exists(Class superclass, Class subclass, DataFlow::ParameterNode selfArg |
superclass = init.getScope() and
subclass = override.getScope() and
subclass = getADirectSubclass+(superclass) and
selfArg.getParameter() = init.getArg(0) and
DataFlow::localFlow(selfArg, self) and
call.calls(self, override.getName()) and
target = superclass.getAMethod() and
target.getName() = override.getName()
)
}
predicate readsFromSelf(Function method) {
exists(DataFlow::ParameterNode self, DataFlow::Node sink |
self.getParameter() = method.getArg(0) and
DataFlow::localFlow(self, sink)
|
sink instanceof DataFlow::ArgumentNode
or
sink = any(DataFlow::AttrRead a).getObject()
)
}
from
Function init, DataFlow::Node self, DataFlow::MethodCallNode call, Function target,
Function override
where
initSelfCallOverridden(init, self, call, target, override) and
readsFromSelf(override) and
not isClassmethod(override) and
not isStaticmethod(override) and
not target.getName().matches("\\_%")
select call, "This call to $@ in an initialization method is overridden by $@.", target,
target.getQualifiedName(), override, override.getQualifiedName()

View File

@@ -0,0 +1,23 @@
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.set_up(arg) # BAD: This method is overridden, so `Sub.set_up` receives a partially initialized instance.
self._state = "OK"
def set_up(self, arg):
"Do some setup"
self.a = 2
class Sub(Super):
def __init__(self, arg):
super().__init__(arg)
self.important_state = "OK"
def set_up(self, arg):
super().set_up(arg)
"Do some more setup"
# BAD: at this point `self._state` is set to `"Not OK"`, and `self.important_state` is not initialized.
if self._state == "OK":
self.b = self.a + 2

View File

@@ -0,0 +1,24 @@
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.super_set_up(arg) # GOOD: This isn't overriden. Instead, additional setup the subclass needs is called by the subclass' `__init__ method.`
self._state = "OK"
def super_set_up(self, arg):
"Do some setup"
self.a = 2
class Sub(Super):
def __init__(self, arg):
super().__init__(arg)
self.sub_set_up(self, arg)
self.important_state = "OK"
def sub_set_up(self, arg):
"Do some more setup"
if self._state == "OK":
self.b = self.a + 2

View File

@@ -1,48 +0,0 @@
#Superclass __init__ calls subclass method
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.set_up(arg)
self._state = "OK"
def set_up(self, arg):
"Do some set up"
class Sub(Super):
def __init__(self, arg):
Super.__init__(self, arg)
self.important_state = "OK"
def set_up(self, arg):
Super.set_up(self, arg)
"Do some more set up" # Dangerous as self._state is "Not OK"
#Improved version with inheritance:
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.super_set_up(arg)
self._state = "OK"
def super_set_up(self, arg):
"Do some set up"
class Sub(Super):
def __init__(self, arg):
Super.__init__(self, arg)
self.sub_set_up(self, arg)
self.important_state = "OK"
def sub_set_up(self, arg):
"Do some more set up"

View File

@@ -1,42 +0,0 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
When an instance of a class is initialized, the super-class state should be
fully initialized before it becomes visible to the subclass.
Calling methods of the subclass in the superclass' <code>__init__</code>
method violates this important invariant.
</p>
</overview>
<recommendation>
<p>Do not use methods that are subclassed in the construction of an object.
For simpler cases move the initialization into the superclass' <code>__init__</code> method,
preventing it being overridden. Additional initialization of subclass should
be done in the <code>__init__</code> method of the subclass.
For more complex cases, it is advisable to use a static method or function to manage
object creation.
</p>
<p>Alternatively, avoid inheritance altogether using composition instead.</p>
</recommendation>
<example>
<sample src="InitCallsSubclassMethod.py" />
</example>
<references>
<li>CERT Secure Coding: <a href="https://www.securecoding.cert.org/confluence/display/java/MET05-J.+Ensure+that+constructors+do+not+call+overridable+methods">
Rule MET05-J</a>. Although this is a Java rule it applies to most object-oriented languages.</li>
<li>Python Standard Library: <a href="http://docs.python.org/library/functions.html#staticmethod">Static methods</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -1,30 +0,0 @@
/**
* @name `__init__` method calls overridden method
* @description Calling a method from `__init__` that is overridden by a subclass may result in a partially
* initialized instance being observed.
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity low
* @precision high
* @id py/init-calls-subclass
*/
import python
from
ClassObject supercls, string method, Call call, FunctionObject overriding,
FunctionObject overridden
where
exists(FunctionObject init, SelfAttribute sa |
supercls.declaredAttribute("__init__") = init and
call.getScope() = init.getFunction() and
call.getFunc() = sa
|
sa.getName() = method and
overridden = supercls.declaredAttribute(method) and
overriding.overrides(overridden)
)
select call, "Call to self.$@ in __init__ method, which is overridden by $@.", overridden, method,
overriding, overriding.descriptiveString()

View File

@@ -2,8 +2,10 @@
* @name Missing call to `__del__` during object destruction
* @description An omitted call to a super-class `__del__` method may lead to class instances not being cleaned up properly.
* @kind problem
* @tags efficiency
* @tags quality
* reliability
* correctness
* performance
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -2,7 +2,8 @@
* @name Missing call to `__init__` during object initialization
* @description An omitted call to a super-class `__init__` method may lead to objects of this class not being fully initialized.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Mutation of descriptor in `__get__` or `__set__` method.
* @description Descriptor objects can be shared across many instances. Mutating them can cause strange side effects or race conditions.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -4,7 +4,8 @@
* method, hides the method in the subclass.
* @kind problem
* @problem.severity error
* @tags maintainability
* @tags quality
* reliability
* correctness
* @sub-severity low
* @precision high

View File

@@ -2,7 +2,8 @@
* @name Multiple calls to `__del__` during object destruction
* @description A duplicated call to a super-class `__del__` method may lead to class instances not be cleaned up properly.
* @kind problem
* @tags efficiency
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Multiple calls to `__init__` during object initialization
* @description A duplicated call to a super-class `__init__` method may lead to objects of this class not being properly initialized.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity high

View File

@@ -4,7 +4,8 @@
* parameter of the __init__ method of the class being
* instantiated, will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-628
* @problem.severity error

View File

@@ -3,7 +3,8 @@
* @description Using too many or too few arguments in a call to the `__init__`
* method of a class will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-685
* @problem.severity error

View File

@@ -2,9 +2,9 @@
* @name Except block handles 'BaseException'
* @description Handling 'BaseException' means that system exits and keyboard interrupts may be mis-handled.
* @kind problem
* @tags reliability
* readability
* convention
* @tags quality
* reliability
* error-handling
* external/cwe/cwe-396
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,8 +2,9 @@
* @name Empty except
* @description Except doesn't do anything and has no comment
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* error-handling
* external/cwe/cwe-390
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,9 +2,9 @@
* @name Non-exception in 'except' clause
* @description An exception handler specifying a non-exception type will never handle any exception.
* @kind problem
* @tags reliability
* correctness
* types
* @tags quality
* reliability
* error-handling
* @problem.severity error
* @sub-severity low
* @precision very-high

View File

@@ -2,9 +2,9 @@
* @name Illegal raise
* @description Raising a non-exception object or type will result in a TypeError being raised instead.
* @kind problem
* @tags reliability
* correctness
* types
* @tags quality
* reliability
* error-handling
* @problem.severity error
* @sub-severity high
* @precision very-high

View File

@@ -3,8 +3,9 @@
* @description Handling general exceptions before specific exceptions means that the specific
* handlers are never executed.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* error-handling
* external/cwe/cwe-561
* @problem.severity error
* @sub-severity low

View File

@@ -6,8 +6,9 @@
* @sub-severity high
* @precision very-high
* @id py/raise-not-implemented
* @tags reliability
* maintainability
* @tags quality
* reliability
* error-handling
*/
import python

View File

@@ -2,9 +2,9 @@
* @name First argument to super() is not enclosing class
* @description Calling super with something other than the enclosing class may cause incorrect object initialization.
* @kind problem
* @tags reliability
* maintainability
* convention
* @tags quality
* reliability
* correctness
* external/cwe/cwe-687
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Comparison of constants
* @description Comparison of constants is always constant, but is harder to read than a simple constant.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-570
* external/cwe/cwe-571

View File

@@ -2,10 +2,9 @@
* @name Comparison of identical values
* @description Comparison of identical values, the intent of which is unclear.
* @kind problem
* @tags reliability
* correctness
* @tags quality
* maintainability
* readability
* convention
* external/cwe/cwe-570
* external/cwe/cwe-571
* @problem.severity warning

View File

@@ -2,8 +2,9 @@
* @name Maybe missing 'self' in comparison
* @description Comparison of identical values, the intent of which is unclear.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-570
* external/cwe/cwe-571
* @problem.severity warning

View File

@@ -2,7 +2,9 @@
* @name Redundant comparison
* @description The result of a comparison is implied by a previous comparison.
* @kind problem
* @tags useless-code
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-561
* external/cwe/cwe-570
* external/cwe/cwe-571

View File

@@ -2,7 +2,8 @@
* @name Membership test with a non-container
* @description A membership test, such as 'item in sequence', with a non-container on the right hand side will raise a 'TypeError'.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Duplicate key in dict literal
* @description Duplicate key in dict literal. All but the last will be lost.
* @kind problem
* @tags reliability
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-561
* @problem.severity warning

View File

@@ -2,8 +2,10 @@
* @name Testing equality to None
* @description Testing whether an object is 'None' using the == operator is inefficient and potentially incorrect.
* @kind problem
* @tags efficiency
* maintainability
* @tags quality
* reliability
* correctness
* performance
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Formatted object is not a mapping
* @description The formatted object must be a mapping when the format includes a named specifier; otherwise a TypeError will be raised."
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name `__del__` is called explicitly
* @description The `__del__` special method is called by the virtual machine when an object is being finalized. It should not be called explicitly.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity low

View File

@@ -3,7 +3,8 @@
* @description Using implicit and explicit numbering in string formatting operations, such as '"{}: {1}".format(a,b)', will raise a ValueError.
* @kind problem
* @problem.severity error
* @tags reliability
* @tags quality
* reliability
* correctness
* @sub-severity low
* @precision high

View File

@@ -2,7 +2,8 @@
* @name Unused argument in a formatting call
* @description Including surplus arguments in a formatting call makes code more difficult to read and may indicate an error.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity warning
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Unused named argument in formatting call
* @description Including surplus keyword arguments in a formatting call makes code more difficult to read and may indicate an error.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity warning
* @sub-severity high

View File

@@ -4,7 +4,8 @@
* where the names of format items in the format string differs from the names of the values to be formatted will raise a KeyError.
* @kind problem
* @problem.severity error
* @tags reliability
* @tags quality
* reliability
* correctness
* @sub-severity low
* @precision high

View File

@@ -3,7 +3,8 @@
* @description A string formatting operation, such as '"{0}: {1}, {2}".format(a,b)',
* where the number of values to be formatted is too few for the format string will raise an IndexError.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Unhashable object hashed
* @description Hashing an object which is not hashable will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Comparison using is when operands support `__eq__`
* @description Comparison using 'is' when equivalence is not the same as identity
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity low

View File

@@ -2,9 +2,9 @@
* @name Non-callable called
* @description A call to an object which is not a callable will raise a TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* types
* @problem.severity error
* @sub-severity high
* @precision high

View File

@@ -3,7 +3,9 @@
* @description Using '\b' to escape the backspace character in a regular expression is confusing
* since it could be mistaken for a word boundary assertion.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* readability
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Duplication in regular expression character class
* @description Duplicate characters in a class have no effect and may indicate an error in the regular expression.
* @kind problem
* @tags reliability
* @tags quality
* maintainability
* readability
* @problem.severity warning
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Missing part of special group in regular expression
* @description Incomplete special groups are parsed as normal groups and are unlikely to match the intended strings.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Unmatchable caret in regular expression
* @description Regular expressions containing a caret '^' in the middle cannot be matched, whatever the input.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Unmatchable dollar in regular expression
* @description Regular expressions containing a dollar '$' in the middle cannot be matched, whatever the input.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,9 +2,9 @@
* @name Implicit string concatenation in a list
* @description Omitting a comma between strings causes implicit concatenation which is confusing in a list.
* @kind problem
* @tags reliability
* @tags quality
* maintainability
* convention
* readability
* external/cwe/cwe-665
* @problem.severity warning
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Unnecessary lambda
* @description A lambda is used that calls through to a function without modifying any parameters
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Unsupported format character
* @description An unsupported format character in a format string
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -4,7 +4,8 @@
* parameter of the called function or method, will result in a
* TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-628
* @problem.severity error

View File

@@ -3,7 +3,8 @@
* @description A string formatting operation, such as '"%s: %s, %s" % (a,b)', where the number of conversion specifiers in the
* format string differs from the number of values to be formatted will raise a TypeError.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-685
* @problem.severity error

View File

@@ -2,7 +2,8 @@
* @name Wrong number of arguments in a call
* @description Using too many or too few arguments in a call to a function will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-685
* @problem.severity error

View File

@@ -2,8 +2,9 @@
* @name Explicit returns mixed with implicit (fall through) returns
* @description Mixing implicit and explicit returns indicates a likely error as implicit returns always return 'None'.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* correctness
* @problem.severity recommendation
* @sub-severity high
* @precision high

View File

@@ -2,7 +2,8 @@
* @name `__init__` method returns a value
* @description Explicitly returning a value from an `__init__` method will raise a TypeError.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,9 +2,9 @@
* @name Non-standard exception raised in special method
* @description Raising a non-standard exception in a special method alters the expected interface of that method.
* @kind problem
* @tags reliability
* maintainability
* convention
* @tags quality
* reliability
* error-handling
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,9 @@
* @name Mismatch between signature and use of an overriding method
* @description Method has a different signature from the overridden method and, if it were called, would be likely to cause an error.
* @kind problem
* @tags maintainability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -3,7 +3,9 @@
* @description Method has a signature that differs from both the signature of its overriding methods and
* the arguments with which it is called, and if it were called, would be likely to cause an error.
* @kind problem
* @tags maintainability
* @tags quality
* reliability
* correctness
* @problem.severity recommendation
* @sub-severity high
* @precision high

View File

@@ -2,7 +2,8 @@
* @name `__init__` method is a generator
* @description `__init__` method is a generator.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name `__iter__` method returns a non-iterator
* @description The `__iter__` method returns a non-iterator which, if used in a 'for' loop, would raise a 'TypeError'.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -3,34 +3,27 @@
"qhelp.dtd">
<qhelp>
<overview>
<p>The <code>__iter__</code> method of an iterator should return self.
This is important so that iterators can be used as sequences in any context
that expect a sequence. To do so requires that <code>__iter__</code> is
idempotent on iterators.</p>
<p>
Note that sequences and mapping should return a new iterator, it is just the returned
iterator that must obey this constraint.
<p>Iterator classes (classes defining a <code>__next__</code> method) should have an <code>__iter__</code> method that returns the iterator itself.
This ensures that the object is also an iterable; and behaves as expected when used anywhere an iterator or iterable is expected, such as in <code>for</code> loops.
</p>
</overview>
<recommendation>
<p>Make the <code>__iter__</code> return self unless the class should not be an iterator,
in which case rename the <code>next</code> (Python 2) or <code>__next__</code> (Python 3)
to something else.</p>
<p>Ensure that the <code>__iter__</code> method returns <code>self</code>, or is otherwise equivalent as an iterator to <code>self</code>.</p>
</recommendation>
<example>
<p>In this example the <code>Counter</code> class's <code>__iter__</code> method does not
return self (or even an iterator). This will cause the program to fail when anyone attempts
to use the iterator in a <code>for</code> loop or <code>in</code> statement.</p>
<sample src="IterReturnsNonSelf.py" />
<p>In the following example, the <code>MyRange</code> class's <code>__iter__</code> method does not return <code>self</code>.
This would lead to unexpected results when used with a <code>for</code> loop or <code>in</code> statement.</p>
<sample src="examples/IterReturnsNonSelf.py" />
</example>
<references>
<li>Python Language Reference: <a href="http://docs.python.org/2.7/reference/datamodel.html#object.__iter__">object.__iter__</a>.</li>
<li>Python Standard Library: <a href="http://docs.python.org/2/library/stdtypes.html#typeiter">Iterators</a>.</li>
<li>Python Language Reference: <a href="http://docs.python.org/3/reference/datamodel.html#object.__iter__">object.__iter__</a>.</li>
<li>Python Standard Library: <a href="http://docs.python.org/3/library/stdtypes.html#typeiter">Iterators</a>.</li>
</references>

View File

@@ -4,6 +4,7 @@
* @kind problem
* @tags reliability
* correctness
* quality
* @problem.severity error
* @sub-severity low
* @precision high
@@ -11,20 +12,79 @@
*/
import python
import semmle.python.ApiGraphs
Function iter_method(ClassValue t) { result = t.lookup("__iter__").(FunctionValue).getScope() }
/** Gets the __iter__ method of `c`. */
Function iterMethod(Class c) { result = c.getAMethod() and result.getName() = "__iter__" }
predicate is_self(Name value, Function f) { value.getVariable() = f.getArg(0).(Name).getVariable() }
/** Gets the `__next__` method of `c`. */
Function nextMethod(Class c) { result = c.getAMethod() and result.getName() = "__next__" }
predicate returns_non_self(Function f) {
exists(f.getFallthroughNode())
/** Holds if `var` is a variable referring to the `self` parameter of `f`. */
predicate isSelfVar(Function f, Name var) { var.getVariable() = f.getArg(0).(Name).getVariable() }
/** Holds if `e` is an expression that an iter function `f` should return. */
predicate isGoodReturn(Function f, Expr e) {
isSelfVar(f, e)
or
exists(Return r | r.getScope() = f and not is_self(r.getValue(), f))
or
exists(Return r | r.getScope() = f and not exists(r.getValue()))
exists(DataFlow::CallCfgNode call, DataFlow::AttrRead read, DataFlow::Node selfNode |
e = call.asExpr()
|
call = API::builtin("iter").getACall() and
call.getArg(0) = read and
read.accesses(selfNode, "__next__") and
isSelfVar(f, selfNode.asExpr()) and
call.getArg(1).asExpr() instanceof None
)
}
from ClassValue t, Function iter
where t.isIterator() and iter = iter_method(t) and returns_non_self(iter)
select t, "Class " + t.getName() + " is an iterator but its $@ method does not return 'self'.",
iter, iter.getName()
/** Holds if the iter method `f` does not return `self` or an equivalent. */
predicate returnsNonSelf(Function f) {
exists(f.getFallthroughNode())
or
exists(Return r | r.getScope() = f and not isGoodReturn(f, r.getValue()))
}
/** Holds if `iter` and `next` methods are wrappers around some field. */
predicate iterWrapperMethods(Function iter, Function next) {
exists(string field |
exists(Return r, DataFlow::Node self, DataFlow::AttrRead read |
r.getScope() = iter and
r.getValue() = [iterCall(read).asExpr(), read.asExpr()] and
read.accesses(self, field) and
isSelfVar(iter, self.asExpr())
) and
exists(Return r, DataFlow::Node self, DataFlow::AttrRead read |
r.getScope() = next and
r.getValue() = nextCall(read).asExpr() and
read.accesses(self, field) and
isSelfVar(next, self.asExpr())
)
)
}
/** Gets a call to `iter(arg)` or `arg.__iter__()`. */
private DataFlow::CallCfgNode iterCall(DataFlow::Node arg) {
result.(DataFlow::MethodCallNode).calls(arg, "__iter__")
or
result = API::builtin("iter").getACall() and
arg = result.getArg(0) and
not exists(result.getArg(1))
}
/** Gets a call to `next(arg)` or `arg.__next__()`. */
private DataFlow::CallCfgNode nextCall(DataFlow::Node arg) {
result.(DataFlow::MethodCallNode).calls(arg, "__next__")
or
result = API::builtin("next").getACall() and
arg = result.getArg(0)
}
from Class c, Function iter, Function next
where
next = nextMethod(c) and
iter = iterMethod(c) and
returnsNonSelf(iter) and
not iterWrapperMethods(iter, next)
select iter, "Iter method of iterator $@ does not return `" + iter.getArg(0).getName() + "`.", c,
c.getName()

View File

@@ -3,8 +3,9 @@
* @description Modifying the default value of a parameter can lead to unexpected
* results.
* @kind path-problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -2,11 +2,9 @@
* @name Overly complex `__del__` method
* @description `__del__` methods may be called at arbitrary times, perhaps never called at all, and should be simple.
* @kind problem
* @tags efficiency
* @tags quality
* maintainability
* complexity
* statistical
* non-attributable
* @problem.severity recommendation
* @sub-severity low
* @precision high

View File

@@ -2,9 +2,9 @@
* @name Returning tuples with varying lengths
* @description A function that potentially returns tuples of different lengths may indicate a problem.
* @kind problem
* @tags reliability
* maintainability
* quality
* @tags quality
* reliability
* correctness
* @problem.severity recommendation
* @sub-severity high
* @precision high

View File

@@ -4,7 +4,8 @@
* number and type of parameters has the potential to cause an error when there is a mismatch.
* @kind problem
* @problem.severity warning
* @tags reliability
* @tags quality
* reliability
* correctness
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,9 @@
* @name Use of the return value of a procedure
* @description The return value of a procedure (a function that does not return a value) is used. This is confusing to the reader as the value (None) has no meaning.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* readability
* @problem.severity warning
* @sub-severity low
* @precision high

View File

@@ -4,10 +4,10 @@ class MyRange(object):
self.high = high
def __iter__(self):
return self.current
return (self.current, self.high) # BAD: does not return `self`.
def next(self):
def __next__(self):
if self.current > self.high:
raise StopIteration
return None
self.current += 1
return self.current - 1

View File

@@ -2,7 +2,8 @@
* @name Encoding error
* @description Encoding errors cause failures at runtime and prevent analysis of the code.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,9 @@
* @name Module is imported with 'import' and 'import from'
* @description A module is imported with the "import" and "import from" statements
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* readability
* @problem.severity recommendation
* @sub-severity low
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Module imports itself
* @description A module imports itself
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Module is imported more than once
* @description Importing a module a second time has no effect and impairs readability
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,7 +2,8 @@
* @name Syntax error
* @description Syntax errors cause failures at runtime and prevent analysis of the code.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity high

View File

@@ -3,8 +3,9 @@
* @description Importing a module using 'import *' may unintentionally pollute the global
* namespace if the module does not define `__all__`
* @kind problem
* @tags maintainability
* modularity
* @tags quality
* maintainability
* readability
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Unused import
* @description Import is not required as it is not used
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity recommendation
* @sub-severity high

View File

@@ -2,9 +2,9 @@
* @name Commented-out code
* @description Commented-out code makes the remaining code more difficult to read.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* readability
* documentation
* @problem.severity recommendation
* @sub-severity high
* @precision high

View File

@@ -2,10 +2,10 @@
* @name File is not always closed
* @description Opening a file without ensuring that it is always closed may lead to data loss or resource leaks.
* @kind problem
* @tags efficiency
* @tags quality
* reliability
* correctness
* resources
* quality
* performance
* external/cwe/cwe-772
* @problem.severity warning
* @sub-severity high

View File

@@ -2,8 +2,9 @@
* @name Asserting a tuple
* @description Using an assert statement to test a tuple provides no validity checking.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* correctness
* external/cwe/cwe-670
* @problem.severity error
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Constant in conditional expression or statement
* @description The conditional is always true or always false
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-561
* external/cwe/cwe-570

View File

@@ -2,9 +2,9 @@
* @name Iterable can be either a string or a sequence
* @description Iteration over either a string or a sequence in the same loop can cause errors that are hard to find.
* @kind problem
* @tags reliability
* maintainability
* non-local
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -3,9 +3,9 @@
* @description Assigning multiple variables without ensuring that you define a
* value for each variable causes an exception at runtime.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* types
* @problem.severity error
* @sub-severity low
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Modification of dictionary returned by locals()
* @description Modifications of the dictionary returned by locals() are not propagated to the local variables of a function.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity low

View File

@@ -3,8 +3,9 @@
* @description Nested loops in which the target variable is the same for each loop make
* the behavior of the loops difficult to understand.
* @kind problem
* @tags maintainability
* correctness
* @tags quality
* maintainability
* readability
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -3,7 +3,8 @@
* @description Redefining a variable in an inner loop and then using
* the variable in an outer loop causes unexpected behavior.
* @kind problem
* @tags maintainability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low

View File

@@ -2,9 +2,9 @@
* @name Non-iterable used in for loop
* @description Using a non-iterable as the object in a 'for' loop causes a TypeError.
* @kind problem
* @tags reliability
* @tags quality
* reliability
* correctness
* types
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -2,8 +2,9 @@
* @name Redundant assignment
* @description Assigning a variable to itself is useless and very likely indicates an error in the code.
* @kind problem
* @tags reliability
* useless-code
* @tags quality
* reliability
* correctness
* external/cwe/cwe-563
* @problem.severity error
* @sub-severity low

View File

@@ -3,9 +3,9 @@
* @description Using a 'try-finally' block to ensure only that a resource is closed makes code more
* difficult to read.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* readability
* convention
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -3,8 +3,9 @@
* @description Side-effects in assert statements result in differences between normal
* and optimized behavior.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high

View File

@@ -2,7 +2,8 @@
* @name Statement has no effect
* @description A statement has no effect
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-561
* @problem.severity recommendation

View File

@@ -2,9 +2,9 @@
* @name Use of a print statement at module level
* @description Using a print statement at module scope (except when guarded by `if __name__ == '__main__'`) will cause surprising output when the module is imported.
* @kind problem
* @tags reliability
* maintainability
* convention
* @tags quality
* reliability
* correctness
* @problem.severity recommendation
* @sub-severity high
* @precision high

View File

@@ -2,7 +2,8 @@
* @name Unnecessary 'else' clause in loop
* @description An 'else' clause in a 'for' or 'while' statement that does not contain a 'break' is redundant.
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity warning
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Unnecessary pass
* @description Unnecessary 'pass' statement
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity warning
* @sub-severity low

View File

@@ -2,7 +2,8 @@
* @name Unreachable code
* @description Code is unreachable
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* external/cwe/cwe-561
* @problem.severity warning

View File

@@ -2,8 +2,9 @@
* @name Unused exception object
* @description An exception object is created, but is not used.
* @kind problem
* @tags reliability
* maintainability
* @tags quality
* reliability
* error-handling
* @problem.severity error
* @sub-severity low
* @precision very-high

View File

@@ -2,7 +2,9 @@
* @name Use of exit() or quit()
* @description exit() or quit() may fail if the interpreter is run with the -S option.
* @kind problem
* @tags maintainability
* @tags quality
* reliability
* correctness
* @problem.severity warning
* @sub-severity low
* @precision very-high

View File

@@ -2,8 +2,9 @@
* @name Imprecise assert
* @description Using 'assertTrue' or 'assertFalse' rather than a more specific assertion can give uninformative failure messages.
* @kind problem
* @tags maintainability
* testability
* @tags quality
* maintainability
* readability
* @problem.severity recommendation
* @sub-severity high
* @precision very-high

View File

@@ -2,7 +2,8 @@
* @name Use of 'global' at module level
* @description Use of the 'global' statement at module level
* @kind problem
* @tags maintainability
* @tags quality
* maintainability
* useless-code
* @problem.severity warning
* @sub-severity low

View File

@@ -2,8 +2,9 @@
* @name Loop variable capture
* @description Capturing a loop variable is not the same as capturing its value, and can lead to unexpected behavior or bugs.
* @kind path-problem
* @tags correctness
* quality
* @tags quality
* reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high

Some files were not shown because too many files have changed in this diff Show More