diff --git a/prompt-injection-summary.md b/prompt-injection-summary.md new file mode 100644 index 00000000000..9250f772e69 --- /dev/null +++ b/prompt-injection-summary.md @@ -0,0 +1,321 @@ +# Prompt Injection Alert Validation Summary + +Validation of the DCA alert-comparison report for the Python queries +`py/system-prompt-injection` (`SystemPromptInjection.ql`) and +`py/user-prompt-injection` (`UserPromptInjection.ql`). + +Source report: +`github/codeql-dca-main` @ `data/prompt-injection-llm-sdks-single` → `reports/alert-comparison.md` + +## Methodology + +Each alert was validated by fetching and reading the actual source code of the target +repository (at the exact commit referenced in the report), inspecting both the reported +**source** (the "user-provided value") and the reported **sink** (the prompt construction), +and confirming the flow. + +Classification rules applied: + +- **TP (true positive):** the source is genuinely untrusted/remote input and the flow really + reaches a prompt of the claimed role (user-role message for `user-prompt-injection`; + system/developer/tool-description for `system-prompt-injection`). By-design / admin flows into a + system prompt count as TP. If only a *convention* (not code) prevents injection, it is a TP. +- **Concern:** the flow is real, but there **is** input validation/sanitization that CodeQL does + not model. Reported as a concern rather than an FP (it is still essentially a true flow). +- **FP (false positive):** static analysis was imprecise — either the flagged flow is implausible + to ever carry attacker input (spurious path / not-really-untrusted source), or, for system-prompt + alerts, the taint did not actually end in a system prompt (sink mis-attribution). + +## Headline metrics + +| Metric | Value | +| --- | --- | +| Total detections | **86** | +| True positives (TP) | **84** | +| Concerns (real flow, unmodeled sanitizer) | **1** | +| False positives (FP) | **1** | +| Precision (TP+Concern treated as real) = 85/86 | **98.8%** | +| False-positive rate = 1/86 | **1.2%** | + +### By query + +| Query | Total | TP | Concern | FP | Precision* | +| --- | :---: | :---: | :---: | :---: | :---: | +| `py/system-prompt-injection` | 3 | 2 | 1 | 0 | 100% | +| `py/user-prompt-injection` | 83 | 82 | 0 | 1 | 98.8% | +| **Total** | **86** | **84** | **1** | **1** | **98.8%** | + +\* Precision counts genuine flows (TP + Concern) as correct; the single Concern is a real flow whose +only mitigation (a Pydantic validator + allowlist) is not modeled by CodeQL. + +### Overall assessment + +Both queries are **highly precise** on this corpus. The `user-prompt-injection` query is marked +`@precision low` in its metadata, yet on real-world LLM-SDK apps essentially every flow it reports is +a genuine, unmediated path from untrusted input (Flask/FastAPI/Django request bodies, webhook +payloads, Gradio/Streamlit widgets) into a user-role LLM message. The `system-prompt-injection` +query (`@precision high`) had no false positives. The only true FP was caused by inter-procedural +conflation of two identically-named `GPT` classes. + +--- + +## System prompt injection (3) + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| S1 | FireBird-Technologies/blog2video `template_studio_llm.py:70` (`system=`) | `template_studio.py:2030,2187` | **Concern** | +| S2 | samuelclay/NewsBlur `utils/ai_functions.py:116` (`system=`) | `apps/analyzer/views.py:377` | **TP** | +| S3 | samuelclay/NewsBlur `utils/ai_functions.py:365` (`system=`) | `apps/analyzer/views.py:377` | **TP** | + +- **S1 — Concern.** The sink is genuinely the Anthropic `system=` parameter. The only user-derived + content reaching it is `layout_id`, embedded as a markdown heading `f"## {layout_id}\n"`. That value + comes from request models where it is either validated by a Pydantic regex `^[a-z][a-z0-9_]*$` or + checked against an allowlist of known layouts (`meta.json`). The free-form `design_doc` / `instruction` + fields do **not** reach `system=` (they flow only to `user=`). The flow is real and correctly points + at a system prompt, but the character-class/allowlist restriction on `layout_id` (which makes practical + injection implausible) is not modeled by CodeQL — hence a **Concern**, not an FP. +- **S2 / S3 — TP.** `prompt = request.POST.get("prompt", "").strip()` (a raw Django POST field) is + interpolated into `system_message = f"""...classification criteria is: {prompt_classifier.prompt}..."""` + and passed as `system=` to `client.messages.create(...)` (text classifier at line 116, vision classifier + at line 365). Only a 500-character length check is applied — no content sanitization. Arbitrary + instructions land directly in the Anthropic system prompt. + +--- + +## User prompt injection (83) + +### FireBird-Technologies/blog2video (2) — both TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U1 | `services/image_gen.py:36` (`images.generate(prompt=…)`) | `routers/projects.py:3392` | **TP** | +| U2 | `services/template_studio_llm.py:71` (`{"role":"user","content":user}`) | `routers/template_studio.py:1048,1111,2030,2187,2679,2738` | **TP** | + +- **U1 — TP.** User-supplied scene-description text flows into the OpenAI image-generation `prompt=` + argument with no content-level sanitization. +- **U2 — TP.** Free-form request-body fields (`instruction` min 5 / max 6000 chars, `design_doc` max + 40000 chars, etc.) flow through helper functions into the Anthropic user-role message. Length caps + only; no content validation. + +### LearningCircuit/local-deep-research (17) — all TP + +All 17 share source `web/api.py:11` col 39–46 = the Flask **`request`** object. Every authenticated POST +endpoint does `query = request.json.get("query")` (only a `isinstance(str)` type-check — no content +sanitization) and passes `query` unchanged through the research pipeline into LLM prompts. + +| # | Sink | Verdict | +| --- | --- | --- | +| U3 | `filters/cross_engine_filter.py:167` — `Query: "{query}"` → `model.invoke(prompt)` | **TP** | +| U4 | `filters/followup_relevance_filter.py:160` — `Follow-up question: "{query}"` | **TP** | +| U5 | `questions/atomic_fact_question.py:79` — `Query: {query}` | **TP** | +| U6 | `questions/atomic_fact_question.py:149` — `Original Query: {original_query}` | **TP** | +| U7 | `questions/browsecomp_question.py:96` — `Query: {query}` | **TP** | +| U8 | `questions/browsecomp_question.py:282` — `Original Query: {query}` | **TP** | +| U9 | `questions/flexible_browsecomp_question.py:61` — `…for: {query}` | **TP** | +| U10 | `questions/standard_question.py:41` — `…answer: {query}` | **TP** | +| U11 | `strategies/langgraph_agent_strategy.py:1146` — `{"role":"user","content":query}` (explicit user role) | **TP** | +| U12 | `strategies/topic_organization_strategy.py:274` — `For the research query: "{query}"` | **TP** | +| U13 | `strategies/topic_organization_strategy.py:909` — refinement prompt w/ original query | **TP** | +| U14 | `strategies/topic_organization_strategy.py:1658` — `RESEARCH QUESTION TO ANSWER: {query}` | **TP** | +| U15 | `strategies/topic_organization_strategy.py:1706` — same `topic_prompt` loop | **TP** | +| U16 | `citation_handlers/base_citation_handler.py:41` — `self.llm.stream(prompt)` | **TP** | +| U17 | `citation_handlers/base_citation_handler.py:83` — `self.llm.invoke(prompt)` | **TP** | +| U18 | `citation_handlers/base_citation_handler.py:91` — `self.llm.invoke(prompt)` | **TP** | +| U19 | `report_generator.py:166` — `Analyze this research content about: {query}` | **TP** | + +### PostHog/posthog (2) — both TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U20 | `user_interviews/backend/max_tools.py:61` | `presentation/webhooks.py:363` | **TP** | +| U21 | `user_interviews/backend/max_tools.py:71` | `presentation/webhooks.py:363` | **TP** | + +Untrusted Vapi `end-of-call-report` webhook fields (`transcript` / `summary`, i.e. the interviewee's +speech) are stored via the ORM and later joined into `interview_summaries_text`, which is placed in the +user-role message of a `gpt-4.1-mini` call. DB-mediated but a genuine indirect-injection vector. + +### Significant-Gravitas/AutoGPT (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U22 | `backend/data/tally.py:411` — `{"role":"user","content":f"{_EXTRACTION_PROMPT}{formatted_text}…"}` | `api/features/v1.py:388` (`OnboardingProfileRequest`) | **TP** | + +User-controlled onboarding-profile fields (`user_name`, `user_role`, `pain_points`) from a FastAPI POST +body are formatted verbatim into the user-role extraction prompt. + +### SocialAI-tianji/Tianji (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U23 | `agents/metagpt_agents/utils/agent_llm.py:106` — `{"role":"user","content":prompt}` | `run/demo_agent_metagpt.py:100` (`st.chat_input()`) | **TP** | + +### aliasrobotics/cai (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U24 | `sdk/agents/items.py:220` — `[{"content":input,"role":"user"}]` | `api/app.py:567,655,709,830` (FastAPI `payload.input`/`payload.prompt`) | **TP** | + +### egolife-ai/Ego-R1 (6) — 5 TP, 1 FP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U25 | `api/rag/r1rag/utils.py:49` (`"content"` user array) | 8 FastAPI `/query` endpoints (`request.keywords`) | **TP** | +| U26 | `api/rag/r1rag/utils.py:52` (`"text": prompt`) | same 8 endpoints | **TP** | +| U27 | `api/visual_tools/egor1_vlm/utils.py:61` (`{"role":"user","content":message}`) | 3 `/vlm` endpoints (`request.question`) | **TP** | +| U28 | `api/visual_tools/egoschema_vlm/utils.py:92` | same 3 `/vlm` endpoints | **TP** | +| U29 | `api/visual_tools/videomme_vlm/utils.py:62` | same 3 `/vlm` endpoints | **TP** | +| U30 | `cott_gen/utils.py:100` | 3 `visual_tools` `/vlm` endpoints | **FP** | + +- **U30 — FP (the only false positive).** The reported taint path is spurious. The `visual_tools` API + handlers only ever instantiate their *own* local `GPT` class (in `api/visual_tools/*/utils.py`). + `cott_gen/` is a separate offline chain-of-thought pipeline with an independently-defined `GPT` class + that is never imported or invoked by any API endpoint. CodeQL conflated the two identically-named + `GPT` classes with matching `chat(self, message, …)` signatures (duck-typed dispatch), producing an + inter-procedural path that has no concrete call chain. **Imprecision — sink not actually reachable + from the source.** + +### ezgisubasi/youtube-rag-assistant (3) — all TP + +Source for all three: `app.py:224` `st.chat_input("Ask about leadership or business...")`. + +| # | Sink | Verdict | +| --- | --- | --- | +| U31 | `src/services/rag_service.py:213` — user query in eval prompt → `llm.invoke` | **TP** | +| U32 | `src/services/rag_service.py:307` — `.format(..., question=question)` → `llm.invoke` | **TP** | +| U33 | `src/services/rag_service.py:323` — web-fallback prompt → `llm.invoke` | **TP** | + +### llnl/open-ai-co-scientist (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U34 | `app/utils.py:57` — `{"role":"user","content":prompt}` | `app.py:492-496,508-511` (Gradio `gr.Textbox` "Research Goal") | **TP** | + +### openai/openai-agents-python (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U35 | `examples/mcp/manager_example/app.py:107` — `Runner.run(..., input=req.input)` | same file:93 (FastAPI `RunRequest.input`) | **TP** | + +Example/demo code, but the flow (HTTP body → agent user turn) is technically a valid injection path. + +### samuelclay/NewsBlur — archive_extension (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U36 | `apps/archive_extension/views.py:1134` — `{"role":"user","content":prompt}` | same file:1063 (`request.POST.get("category")`) | **TP** | + +The `category` POST field (only `.strip()` applied) is interpolated into the user-role Claude message. + +### showlab/computer_use_ootb (2) — both TP + +Sources: Gradio inputs `app.py:248` / `app.py:598`. + +| # | Sink | Verdict | +| --- | --- | --- | +| U37 | `computer_use_demo/gui_agent/actor/uitars_agent.py:59` — `{"type":"text","text":task}` in user role | **TP** | +| U38 | `computer_use_demo/gui_agent/actor/uitars_agent.py:60` — same user-role content array | **TP** | + +### suki0dayo/AI_film_studio (3) — all TP + +Source for all three: `app.py:6` (`from flask import ... request`) — the standard CodeQL Flask taint +origin node; the concrete untrusted value is `request.json['user_prompt']` in the `/api/llm` POST +handler. (Line 6 is the request import node, **not** a constant — not an FP.) + +| # | Sink | Verdict | +| --- | --- | --- | +| U39 | `app.py:488` — `{'role':'user','parts':[{'text':user_prompt}]}` (Gemini) | **TP** | +| U40 | `app.py:540` — `{'role':'user','content':user_prompt}` (OpenAI-compat) | **TP** | +| U41 | `app.py:546` — outbound `/chat/completions` POST carrying that user-role message | **TP** | + +### truera/trulens (1) — TP + +| # | Sink | Source | Verdict | +| --- | --- | --- | --- | +| U42 | `examples/.../openai_agent_sdk_snowflake_tools/src/agent/app.py:155` — `Runner.run_sync(support_agent, question)` | `.../server.py:78` (FastAPI `ChatRequest.message`) | **TP** | + +Example/expositional code, but the HTTP-body → agent user-turn flow is a valid injection path. + +### xusenlinzy/api-for-open-llm (2) — both TP + +Source for both: `streamlit_app.py:36` `st.chat_input("What is up?")`. + +| # | Sink | Verdict | +| --- | --- | --- | +| U43 | `streamlit-demo/.../multimodal_chat/streamlit_app.py:44` — `{"role":"user","content":[{"type":"text","text":prompt}]}` | **TP** | +| U44 | `streamlit-demo/.../multimodal_chat/streamlit_app.py:47` — `"text": prompt` in same user-role array | **TP** | + +### mdbabumiamssm/LLMs-Universal-Life-Science-and-Clinical-Skills- (38) — all TP + +Vendored `awesome-llm-apps` demos under +`Skills/External_Collections/awesome-llm-apps/`. Every alert is a Streamlit widget +(`st.text_input` / `st.text_area` / `st.chat_input`) flowing unmodified into a user-role LLM message +(`{"role":"user",...}`, `HumanMessage(...)`, `('human', ...)`, or `Runner.run(agent, user_input)`). +No sanitization or allowlisting exists in any of these files. + +| # | Sink file:line | Source | Verdict | +| --- | --- | --- | --- | +| U45 | `.../ai_3dpygame_r1/ai_3dpygame_r1.py:91` | `st.text_area` :58 | **TP** | +| U46 | `.../ai_customer_support_agent/customer_support_agent.py:67` | `st.chat_input` :192 | **TP** | +| U47 | `.../ai_deep_research_agent/deep_research_openai.py:140` | `st.text_input` :57 | **TP** | +| U48 | `.../ai_deep_research_agent/deep_research_openai.py:159` | `st.text_input` :57 | **TP** | +| U49 | `.../ai_system_architect_r1/ai_system_architect_r1.py:201` | `st.chat_input` :302 | **TP** | +| U50 | `.../ai_travel_agent_memory/travel_agent_memory.py:95` | `st.chat_input` :71 | **TP** | +| U51 | `.../llm_app_personalized_memory/llm_app_memory.py:61` | `st.text_input` :42 | **TP** | +| U52 | `.../multi_llm_memory/multi_llm_memory.py:75` | `st.text_input` :57 | **TP** | +| U53 | `.../1_starter_agent/app.py:110` | `st.chat_input` :99 | **TP** | +| U54 | `.../1_starter_agent/app.py:117` | `st.chat_input` :99 | **TP** | +| U55 | `.../1_starter_agent/app.py:129` | `st.chat_input` :99 | **TP** | +| U56 | `.../4_running_agents/agent_runner.py:173` | `st.text_input` :165 | **TP** | +| U57 | `.../4_running_agents/agent_runner.py:196` | `st.text_input` :188 | **TP** | +| U58 | `.../4_running_agents/agent_runner.py:227` | `st.text_input` :211 | **TP** | +| U59 | `.../4_running_agents/agent_runner.py:266` | `st.text_input` :256 | **TP** | +| U60 | `.../4_running_agents/agent_runner.py:309` | `st.text_input` :298 | **TP** | +| U61 | `.../4_running_agents/agent_runner.py:378` | `st.text_input` :361 | **TP** | +| U62 | `.../4_running_agents/agent_runner.py:427` | `st.text_input` :407 | **TP** | +| U63 | `.../4_running_agents/agent_runner.py:480` | `st.text_input` :459 | **TP** | +| U64 | `.../4_running_agents/agent_runner.py:536` | `st.text_input` :512 | **TP** | +| U65 | `.../4_running_agents/agent_runner.py:613` | `st.text_input` :604 | **TP** | +| U66 | `.../4_running_agents/agent_runner.py:635` | `st.text_input` :629 | **TP** | +| U67 | `.../7_sessions/streamlit_sessions_app.py:158` | `st.text_input` :152 | **TP** | +| U68 | `.../7_sessions/streamlit_sessions_app.py:187` | `st.text_input` :181 | **TP** | +| U69 | `.../7_sessions/streamlit_sessions_app.py:219` | `st.text_input` :213 | **TP** | +| U70 | `.../7_sessions/streamlit_sessions_app.py:307` | `st.text_input` :301 | **TP** | +| U71 | `.../7_sessions/streamlit_sessions_app.py:327` | `st.text_input` :321 | **TP** | +| U72 | `.../7_sessions/streamlit_sessions_app.py:352` | `st.text_input` :346 | **TP** | +| U73 | `.../7_sessions/streamlit_sessions_app.py:366` | `st.text_input` :360 | **TP** | +| U74 | `.../7_sessions/streamlit_sessions_app.py:391` | `st.text_input` :385 | **TP** | +| U75 | `.../hybrid_search_rag/main.py:123` | `st.chat_input` :190 | **TP** | +| U76 | `.../llama3.1_local_rag/llama3.1_local_rag.py:53` | `st.text_input` :85 | **TP** | +| U77 | `.../rag_agent_cohere/rag_agent_cohere.py:237` | `st.chat_input` :279 | **TP** | +| U78 | `.../rag_database_routing/rag_database_routing.py:292` | `st.text_input` :376 | **TP** | +| U79 | `.../rag-as-a-service/rag_app.py:102` | `st.text_input` :185 | **TP** | +| U80 | `.../opeani_research_agent/research_agent.py:196` | `st.text_input` :143 | **TP** | +| U81 | `.../customer_support_voice_agent/customer_support_voice_agent.py:290` | `st.text_input` :349 | **TP** | +| U82 | `.../voice_rag_openaisdk/rag_voice.py:248` | `st.text_input` :359 | **TP** | + +--- + +## Concerns (unmodeled mitigations) + +- **S1 (FireBird blog2video, `system=` at `template_studio_llm.py:70`).** Real flow into a system + prompt, but the only user-derived component (`layout_id`) is constrained by a Pydantic regex + `^[a-z][a-z0-9_]*$` and/or an allowlist of known layout ids, which CodeQL does not model. Practical + injection is unlikely, but the query is technically correct to flag the flow. If desired, such regex + allowlist validators could be added as barriers to the `SystemPromptInjection` sanitizer set. + +Additional observations that are **not** downgraded (still TP): + +- Several NewsBlur / AutoGPT flows apply only a *length* check (e.g. `len(prompt) > 500`) — this is not + content sanitization and does not neutralize injection. +- PostHog (U20/U21) is a DB-mediated indirect-injection flow (voice transcript persisted, then re-read + into a prompt) — a legitimate, if less obvious, injection vector. +- `openai/openai-agents-python` (U35) and `truera/trulens` (U42) are example/demo apps; classification + reflects the technical validity of the flow, independent of the code's demo status. + +## False positives (1) + +- **U30 (egolife-ai/Ego-R1, `cott_gen/utils.py:100`).** Spurious inter-procedural path caused by + conflating two distinct classes both named `GPT` with identical `chat(self, message, …)` signatures. + The `visual_tools` API endpoints never call the `cott_gen` `GPT`; no concrete call chain exists, so + the sink is not actually reachable from the reported source. This is a genuine static-analysis + precision issue (duck-typed method-name/signature conflation). diff --git a/python/ql/lib/semmle/python/frameworks/GoogleGenAI.qll b/python/ql/lib/semmle/python/frameworks/GoogleGenAI.qll index 6f679d8eada..279684e7851 100644 --- a/python/ql/lib/semmle/python/frameworks/GoogleGenAI.qll +++ b/python/ql/lib/semmle/python/frameworks/GoogleGenAI.qll @@ -16,7 +16,7 @@ private import semmle.python.ApiGraphs module GoogleGenAI { /** Gets a reference to a `google.genai.Client` instance. */ private API::Node clientRef() { - result = API::moduleImport("google.genai").getMember("Client").getReturn() + result = API::moduleImport("google").getMember("genai").getMember("Client").getReturn() } /** Gets the content dictionaries passed to `models.generate_content`/`generate_content_stream`. */ diff --git a/python/ql/lib/semmle/python/frameworks/google-genai.model.yml b/python/ql/lib/semmle/python/frameworks/google-genai.model.yml index bdbb7ca85aa..06f43af440e 100644 --- a/python/ql/lib/semmle/python/frameworks/google-genai.model.yml +++ b/python/ql/lib/semmle/python/frameworks/google-genai.model.yml @@ -4,10 +4,12 @@ extensions: extensible: sinkModel data: # `system_instruction` on the generation config is a system-level prompt - - ['google.genai', 'Member[types].Member[GenerateContentConfig].Argument[system_instruction:]', 'system-prompt-injection'] + - ['google', 'Member[genai].Member[types].Member[GenerateContentConfig].Argument[system_instruction:]', 'system-prompt-injection'] + # The Live API connect config carries a system instruction + - ['google', 'Member[genai].Member[types].Member[LiveConnectConfig].Argument[system_instruction:]', 'system-prompt-injection'] # Cached content carries a system instruction and user content - - ['google.genai', 'Member[types].Member[CreateCachedContentConfig].Argument[system_instruction:]', 'system-prompt-injection'] - - ['google.genai', 'Member[types].Member[CreateCachedContentConfig].Argument[contents:]', 'user-prompt-injection'] + - ['google', 'Member[genai].Member[types].Member[CreateCachedContentConfig].Argument[system_instruction:]', 'system-prompt-injection'] + - ['google', 'Member[genai].Member[types].Member[CreateCachedContentConfig].Argument[contents:]', 'user-prompt-injection'] # User-level content - ['GoogleGenAI', 'Member[models].Member[generate_content,generate_content_stream].Argument[contents:]', 'user-prompt-injection'] - ['GoogleGenAI', 'Member[models].Member[generate_images,generate_videos,edit_image].Argument[prompt:]', 'user-prompt-injection'] @@ -18,4 +20,4 @@ extensions: pack: codeql/python-all extensible: typeModel data: - - ['GoogleGenAI', 'google.genai', 'Member[Client].ReturnValue'] + - ['GoogleGenAI', 'google', 'Member[genai].Member[Client].ReturnValue'] diff --git a/python/ql/lib/semmle/python/frameworks/langchain.model.yml b/python/ql/lib/semmle/python/frameworks/langchain.model.yml index aef70cc0f3d..220fd1b75a1 100644 --- a/python/ql/lib/semmle/python/frameworks/langchain.model.yml +++ b/python/ql/lib/semmle/python/frameworks/langchain.model.yml @@ -5,17 +5,28 @@ extensions: data: # Message constructors. The first positional argument or the `content` keyword # carries the message text. - - ['langchain_core.messages', 'Member[SystemMessage].Argument[0]', 'system-prompt-injection'] - - ['langchain_core.messages', 'Member[SystemMessage].Argument[content:]', 'system-prompt-injection'] - - ['langchain.schema', 'Member[SystemMessage].Argument[0]', 'system-prompt-injection'] - - ['langchain.schema', 'Member[SystemMessage].Argument[content:]', 'system-prompt-injection'] - - ['langchain_core.messages', 'Member[HumanMessage].Argument[0]', 'user-prompt-injection'] - - ['langchain_core.messages', 'Member[HumanMessage].Argument[content:]', 'user-prompt-injection'] - - ['langchain.schema', 'Member[HumanMessage].Argument[0]', 'user-prompt-injection'] - - ['langchain.schema', 'Member[HumanMessage].Argument[content:]', 'user-prompt-injection'] + - ['langchain_core', 'Member[messages].Member[SystemMessage].Argument[0]', 'system-prompt-injection'] + - ['langchain_core', 'Member[messages].Member[SystemMessage].Argument[content:]', 'system-prompt-injection'] + - ['langchain', 'Member[schema].Member[SystemMessage].Argument[0]', 'system-prompt-injection'] + - ['langchain', 'Member[schema].Member[SystemMessage].Argument[content:]', 'system-prompt-injection'] + - ['langchain_core', 'Member[messages].Member[HumanMessage].Argument[0]', 'user-prompt-injection'] + - ['langchain_core', 'Member[messages].Member[HumanMessage].Argument[content:]', 'user-prompt-injection'] + - ['langchain', 'Member[schema].Member[HumanMessage].Argument[0]', 'user-prompt-injection'] + - ['langchain', 'Member[schema].Member[HumanMessage].Argument[content:]', 'user-prompt-injection'] # Invoking a chat model with user input. - ['LangChainChatModel', 'Member[invoke,stream,predict,call].Argument[0]', 'user-prompt-injection'] - ['LangChainChatModel', 'Member[batch].Argument[0].ListElement', 'user-prompt-injection'] + - ['LangChainChatModel', 'Member[generate].Argument[0].ListElement.ListElement', 'user-prompt-injection'] + # Prompt templates. User input embedded directly into a template. + - ['langchain_core', 'Member[prompts].Member[PromptTemplate].Instance.Member[format].Argument[any-named]', 'user-prompt-injection'] + # Legacy `LLMChain` and `AgentExecutor` take the user input in the `input` field. + - ['LangChainLLMChain', 'Member[invoke].Argument[0].DictionaryElement[input]', 'user-prompt-injection'] + - ['LangChainLLMChain', 'Member[run].Argument[0]', 'user-prompt-injection'] + - ['LangChainAgentExecutor', 'Member[invoke].Argument[0].DictionaryElement[input]', 'user-prompt-injection'] + # The `system_prompt` passed to `create_agent` is a system-level prompt. + - ['langchain', 'Member[agents].Member[create_agent].Argument[system_prompt:]', 'system-prompt-injection'] + # The messages passed to a `create_agent` graph are user-level content. + - ['LangChainAgent', 'Member[invoke,stream].Argument[0].DictionaryElement[messages].ListElement.DictionaryElement[content]', 'user-prompt-injection'] - addsTo: pack: codeql/python-all @@ -29,3 +40,14 @@ extensions: - ['LangChainChatModel', 'langchain_cohere', 'Member[ChatCohere].ReturnValue'] - ['LangChainChatModel', 'langchain_ollama', 'Member[ChatOllama].ReturnValue'] - ['LangChainChatModel', 'langchain_aws', 'Member[ChatBedrock,ChatBedrockConverse].ReturnValue'] + - ['LangChainChatModel', 'langchain_fireworks', 'Member[ChatFireworks].ReturnValue'] + - ['LangChainChatModel', 'langchain_together', 'Member[ChatTogether].ReturnValue'] + - ['LangChainChatModel', 'langchain_xai', 'Member[ChatXAI].ReturnValue'] + - ['LangChainChatModel', 'langchain', 'Member[chat_models].Member[init_chat_model].ReturnValue'] + - ['LangChainLLMChain', 'langchain', 'Member[chains].Member[LLMChain].ReturnValue'] + - ['LangChainLLMChain', 'langchain_classic', 'Member[chains].Member[LLMChain].ReturnValue'] + - ['LangChainAgentExecutor', 'langchain', 'Member[agents].Member[AgentExecutor].ReturnValue'] + - ['LangChainAgentExecutor', 'langchain_classic', 'Member[agents].Member[AgentExecutor].ReturnValue'] + - ['LangChainAgentExecutor', 'langchain', 'Member[agents].Member[AgentExecutor].Member[from_agent_and_tools].ReturnValue'] + - ['LangChainAgentExecutor', 'langchain_classic', 'Member[agents].Member[AgentExecutor].Member[from_agent_and_tools].ReturnValue'] + - ['LangChainAgent', 'langchain', 'Member[agents].Member[create_agent].ReturnValue'] diff --git a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/SystemPromptInjection.expected b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/SystemPromptInjection.expected index efae5073767..10e8525b4d3 100644 --- a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/SystemPromptInjection.expected +++ b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/SystemPromptInjection.expected @@ -10,6 +10,12 @@ | anthropic_test.py:45:16:45:37 | ControlFlowNode for BinaryExpr | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:45:16:45:37 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | anthropic_test.py:57:16:57:37 | ControlFlowNode for BinaryExpr | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:57:16:57:37 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | anthropic_test.py:63:16:63:37 | ControlFlowNode for BinaryExpr | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:63:16:63:37 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:21:33:21:49 | ControlFlowNode for BinaryExpr | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:21:33:21:49 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:35:32:35:53 | ControlFlowNode for BinaryExpr | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:35:32:35:53 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:43:32:43:53 | ControlFlowNode for BinaryExpr | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:43:32:43:53 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:56:32:56:53 | ControlFlowNode for BinaryExpr | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:56:32:56:53 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:17:35:17:58 | ControlFlowNode for BinaryExpr | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:17:35:17:58 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:30:43:30:66 | ControlFlowNode for BinaryExpr | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:30:43:30:66 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:17:22:17:46 | ControlFlowNode for BinaryExpr | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:17:22:17:46 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:22:22:22:46 | ControlFlowNode for BinaryExpr | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:22:22:22:46 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:26:28:26:51 | ControlFlowNode for BinaryExpr | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:26:28:26:51 | ControlFlowNode for BinaryExpr | This system prompt depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | @@ -47,6 +53,30 @@ edges | anthropic_test.py:11:15:11:21 | ControlFlowNode for request | anthropic_test.py:11:15:11:26 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | anthropic_test.py:11:15:11:26 | ControlFlowNode for Attribute | anthropic_test.py:11:15:11:41 | ControlFlowNode for Attribute() | provenance | dict.get | | anthropic_test.py:11:15:11:41 | ControlFlowNode for Attribute() | anthropic_test.py:11:5:11:11 | ControlFlowNode for persona | provenance | | +| gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:3:26:3:32 | ControlFlowNode for request | provenance | | +| gemini_test.py:3:26:3:32 | ControlFlowNode for request | gemini_test.py:11:15:11:21 | ControlFlowNode for request | provenance | | +| gemini_test.py:3:26:3:32 | ControlFlowNode for request | gemini_test.py:51:15:51:21 | ControlFlowNode for request | provenance | | +| gemini_test.py:11:5:11:11 | ControlFlowNode for persona | gemini_test.py:21:33:21:49 | ControlFlowNode for BinaryExpr | provenance | | +| gemini_test.py:11:5:11:11 | ControlFlowNode for persona | gemini_test.py:35:32:35:53 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:13 | +| gemini_test.py:11:5:11:11 | ControlFlowNode for persona | gemini_test.py:43:32:43:53 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:12 | +| gemini_test.py:11:15:11:21 | ControlFlowNode for request | gemini_test.py:11:15:11:26 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| gemini_test.py:11:15:11:26 | ControlFlowNode for Attribute | gemini_test.py:11:15:11:41 | ControlFlowNode for Attribute() | provenance | dict.get | +| gemini_test.py:11:15:11:41 | ControlFlowNode for Attribute() | gemini_test.py:11:5:11:11 | ControlFlowNode for persona | provenance | | +| gemini_test.py:51:5:51:11 | ControlFlowNode for persona | gemini_test.py:56:32:56:53 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:14 | +| gemini_test.py:51:15:51:21 | ControlFlowNode for request | gemini_test.py:51:15:51:26 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| gemini_test.py:51:15:51:26 | ControlFlowNode for Attribute | gemini_test.py:51:15:51:41 | ControlFlowNode for Attribute() | provenance | dict.get | +| gemini_test.py:51:15:51:41 | ControlFlowNode for Attribute() | gemini_test.py:51:5:51:11 | ControlFlowNode for persona | provenance | | +| langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:3:26:3:32 | ControlFlowNode for request | provenance | | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:10:15:10:21 | ControlFlowNode for request | provenance | | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:28:15:28:21 | ControlFlowNode for request | provenance | | +| langchain_test.py:10:5:10:11 | ControlFlowNode for persona | langchain_test.py:17:35:17:58 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:16 | +| langchain_test.py:10:15:10:21 | ControlFlowNode for request | langchain_test.py:10:15:10:26 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| langchain_test.py:10:15:10:26 | ControlFlowNode for Attribute | langchain_test.py:10:15:10:41 | ControlFlowNode for Attribute() | provenance | dict.get | +| langchain_test.py:10:15:10:41 | ControlFlowNode for Attribute() | langchain_test.py:10:5:10:11 | ControlFlowNode for persona | provenance | | +| langchain_test.py:28:5:28:11 | ControlFlowNode for persona | langchain_test.py:30:43:30:66 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:15 | +| langchain_test.py:28:15:28:21 | ControlFlowNode for request | langchain_test.py:28:15:28:26 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| langchain_test.py:28:15:28:26 | ControlFlowNode for Attribute | langchain_test.py:28:15:28:41 | ControlFlowNode for Attribute() | provenance | dict.get | +| langchain_test.py:28:15:28:41 | ControlFlowNode for Attribute() | langchain_test.py:28:5:28:11 | ControlFlowNode for persona | provenance | | | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:2:26:2:32 | ControlFlowNode for request | provenance | | | openai_test.py:2:26:2:32 | ControlFlowNode for request | openai_test.py:12:15:12:21 | ControlFlowNode for request | provenance | | | openai_test.py:12:5:12:11 | ControlFlowNode for persona | openai_test.py:17:22:17:46 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:6 | @@ -79,6 +109,11 @@ models | 9 | Sink: agents; Member[Agent].Argument[instructions:]; system-prompt-injection | | 10 | Sink: agents; Member[Agent].ReturnValue.Member[as_tool].Argument[1,tool_description:]; system-prompt-injection | | 11 | Sink: agents; Member[FunctionTool].Argument[description:]; system-prompt-injection | +| 12 | Sink: google; Member[genai].Member[types].Member[CreateCachedContentConfig].Argument[system_instruction:]; system-prompt-injection | +| 13 | Sink: google; Member[genai].Member[types].Member[GenerateContentConfig].Argument[system_instruction:]; system-prompt-injection | +| 14 | Sink: google; Member[genai].Member[types].Member[LiveConnectConfig].Argument[system_instruction:]; system-prompt-injection | +| 15 | Sink: langchain; Member[agents].Member[create_agent].Argument[system_prompt:]; system-prompt-injection | +| 16 | Sink: langchain_core; Member[messages].Member[SystemMessage].Argument[content:]; system-prompt-injection | nodes | agent_test.py:2:26:2:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | | agent_test.py:2:26:2:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | @@ -107,6 +142,32 @@ nodes | anthropic_test.py:45:16:45:37 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | | anthropic_test.py:57:16:57:37 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | | anthropic_test.py:63:16:63:37 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | +| gemini_test.py:3:26:3:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| gemini_test.py:11:5:11:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | +| gemini_test.py:11:15:11:21 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| gemini_test.py:11:15:11:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| gemini_test.py:11:15:11:41 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| gemini_test.py:21:33:21:49 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:35:32:35:53 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:43:32:43:53 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:51:5:51:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | +| gemini_test.py:51:15:51:21 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| gemini_test.py:51:15:51:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| gemini_test.py:51:15:51:41 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| gemini_test.py:56:32:56:53 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:10:5:10:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | +| langchain_test.py:10:15:10:21 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:10:15:10:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| langchain_test.py:10:15:10:41 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:17:35:17:58 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| langchain_test.py:28:5:28:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | +| langchain_test.py:28:15:28:21 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:28:15:28:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| langchain_test.py:28:15:28:41 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:30:43:30:66 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | | openai_test.py:2:26:2:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | | openai_test.py:12:5:12:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | @@ -130,10 +191,3 @@ nodes | openrouter_test.py:18:28:18:51 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | | openrouter_test.py:29:22:29:45 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | subpaths -testFailures -| gemini_test.py:3:35:3:44 | Comment # $ Source | Missing result: Source | -| gemini_test.py:21:52:21:88 | Comment # $ Alert[py/system-prompt-injection] | Missing result: Alert[py/system-prompt-injection] | -| gemini_test.py:35:57:35:93 | Comment # $ Alert[py/system-prompt-injection] | Missing result: Alert[py/system-prompt-injection] | -| gemini_test.py:43:57:43:93 | Comment # $ Alert[py/system-prompt-injection] | Missing result: Alert[py/system-prompt-injection] | -| langchain_test.py:3:35:3:44 | Comment # $ Source | Missing result: Source | -| langchain_test.py:17:63:17:99 | Comment # $ Alert[py/system-prompt-injection] | Missing result: Alert[py/system-prompt-injection] | diff --git a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/gemini_test.py b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/gemini_test.py index 8396df3e178..e1c2219921b 100644 --- a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/gemini_test.py +++ b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/gemini_test.py @@ -44,3 +44,16 @@ def get_input_gemini(): ), ) print(cache) + + +@app.route("/gemini-live") +async def get_input_gemini_live(): + persona = request.args.get("persona") + + async with client.aio.live.connect( + model="gemini-2.0-flash", + config=types.LiveConnectConfig( + system_instruction="Talk like " + persona, # $ Alert[py/system-prompt-injection] + ), + ) as session: + print(session) diff --git a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/langchain_test.py b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/langchain_test.py index 468b561326e..baf64eb7b0b 100644 --- a/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/langchain_test.py +++ b/python/ql/test/query-tests/Security/CWE-1427-SystemPromptInjection/langchain_test.py @@ -19,3 +19,12 @@ def get_input_langchain(): ] ) print(result) + + +@app.route("/langchain-create-agent") +def get_input_langchain_create_agent(): + from langchain.agents import create_agent + + persona = request.args.get("persona") + + create_agent("gpt-4.1", system_prompt="Talk like a " + persona) # $ Alert[py/system-prompt-injection] diff --git a/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/UserPromptInjection.expected b/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/UserPromptInjection.expected index 3926153fe59..b50b01633c4 100644 --- a/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/UserPromptInjection.expected +++ b/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/UserPromptInjection.expected @@ -4,7 +4,24 @@ | agent_test.py:20:28:20:32 | ControlFlowNode for query | agent_test.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_test.py:20:28:20:32 | ControlFlowNode for query | This prompt construction depends on a $@. | agent_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | anthropic_test.py:20:28:20:32 | ControlFlowNode for query | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:20:28:20:32 | ControlFlowNode for query | This prompt construction depends on a $@. | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | anthropic_test.py:29:16:29:55 | ControlFlowNode for BinaryExpr | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:29:16:29:55 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:15:18:15:22 | ControlFlowNode for query | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:15:18:15:22 | ControlFlowNode for query | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:20:18:29:9 | ControlFlowNode for List | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:20:18:29:9 | ControlFlowNode for List | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:25:33:25:37 | ControlFlowNode for query | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:25:33:25:37 | ControlFlowNode for query | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:33:35:33:58 | ControlFlowNode for BinaryExpr | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:33:35:33:58 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:37:16:37:20 | ControlFlowNode for query | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:37:16:37:20 | ControlFlowNode for query | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| gemini_test.py:43:22:43:26 | ControlFlowNode for query | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:43:22:43:26 | ControlFlowNode for query | This prompt construction depends on a $@. | gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:17:34:17:38 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:17:34:17:38 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | | langchain_test.py:21:28:21:51 | ControlFlowNode for BinaryExpr | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:21:28:21:51 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:34:28:34:32 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:34:28:34:32 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:35:27:35:31 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:35:27:35:31 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:36:22:36:26 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:36:22:36:26 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:37:39:37:43 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:37:39:37:43 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:40:22:40:26 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:40:22:40:26 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:50:30:50:34 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:50:30:50:34 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:61:28:61:32 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:61:28:61:32 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:62:15:62:19 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:62:15:62:19 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:65:31:65:35 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:65:31:65:35 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | +| langchain_test.py:68:60:68:64 | ControlFlowNode for query | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:68:60:68:64 | ControlFlowNode for query | This prompt construction depends on a $@. | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:16:15:16:19 | ControlFlowNode for query | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:16:15:16:19 | ControlFlowNode for query | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:20:15:29:9 | ControlFlowNode for List | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:20:15:29:9 | ControlFlowNode for List | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | | openai_test.py:27:28:27:32 | ControlFlowNode for query | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:27:28:27:32 | ControlFlowNode for query | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value | @@ -20,13 +37,13 @@ edges | agent_test.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_test.py:2:26:2:32 | ControlFlowNode for request | provenance | | | agent_test.py:2:26:2:32 | ControlFlowNode for request | agent_test.py:9:13:9:19 | ControlFlowNode for request | provenance | | -| agent_test.py:9:5:9:9 | ControlFlowNode for query | agent_test.py:13:38:13:42 | ControlFlowNode for query | provenance | Sink:MaD:9 | +| agent_test.py:9:5:9:9 | ControlFlowNode for query | agent_test.py:13:38:13:42 | ControlFlowNode for query | provenance | Sink:MaD:17 | | agent_test.py:9:5:9:9 | ControlFlowNode for query | agent_test.py:20:28:20:32 | ControlFlowNode for query | provenance | | | agent_test.py:9:5:9:9 | ControlFlowNode for query | agent_test.py:20:28:20:32 | ControlFlowNode for query | provenance | | | agent_test.py:9:13:9:19 | ControlFlowNode for request | agent_test.py:9:13:9:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | agent_test.py:9:13:9:24 | ControlFlowNode for Attribute | agent_test.py:9:13:9:37 | ControlFlowNode for Attribute() | provenance | dict.get | | agent_test.py:9:13:9:37 | ControlFlowNode for Attribute() | agent_test.py:9:5:9:9 | ControlFlowNode for query | provenance | | -| agent_test.py:18:13:21:13 | ControlFlowNode for Dict [Dictionary element at key content] | agent_test.py:17:15:22:9 | ControlFlowNode for List | provenance | Sink:MaD:10 Sink:MaD:10 | +| agent_test.py:18:13:21:13 | ControlFlowNode for Dict [Dictionary element at key content] | agent_test.py:17:15:22:9 | ControlFlowNode for List | provenance | Sink:MaD:18 Sink:MaD:18 | | agent_test.py:20:28:20:32 | ControlFlowNode for query | agent_test.py:18:13:21:13 | ControlFlowNode for Dict [Dictionary element at key content] | provenance | | | anthropic_test.py:2:26:2:32 | ControlFlowNode for ImportMember | anthropic_test.py:2:26:2:32 | ControlFlowNode for request | provenance | | | anthropic_test.py:2:26:2:32 | ControlFlowNode for request | anthropic_test.py:10:15:10:21 | ControlFlowNode for request | provenance | | @@ -37,12 +54,52 @@ edges | anthropic_test.py:11:13:11:19 | ControlFlowNode for request | anthropic_test.py:11:13:11:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | anthropic_test.py:11:13:11:24 | ControlFlowNode for Attribute | anthropic_test.py:11:13:11:37 | ControlFlowNode for Attribute() | provenance | dict.get | | anthropic_test.py:11:13:11:37 | ControlFlowNode for Attribute() | anthropic_test.py:11:5:11:9 | ControlFlowNode for query | provenance | | +| gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | gemini_test.py:3:26:3:32 | ControlFlowNode for request | provenance | | +| gemini_test.py:3:26:3:32 | ControlFlowNode for request | gemini_test.py:11:13:11:19 | ControlFlowNode for request | provenance | | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:15:18:15:22 | ControlFlowNode for query | provenance | Sink:MaD:3 | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:25:33:25:37 | ControlFlowNode for query | provenance | | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:25:33:25:37 | ControlFlowNode for query | provenance | | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:33:35:33:58 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:2 | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:37:16:37:20 | ControlFlowNode for query | provenance | Sink:MaD:4 | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | gemini_test.py:43:22:43:26 | ControlFlowNode for query | provenance | Sink:MaD:19 | +| gemini_test.py:11:13:11:19 | ControlFlowNode for request | gemini_test.py:11:13:11:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| gemini_test.py:11:13:11:24 | ControlFlowNode for Attribute | gemini_test.py:11:13:11:37 | ControlFlowNode for Attribute() | provenance | dict.get | +| gemini_test.py:11:13:11:37 | ControlFlowNode for Attribute() | gemini_test.py:11:5:11:9 | ControlFlowNode for query | provenance | | +| gemini_test.py:21:13:28:13 | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | gemini_test.py:20:18:29:9 | ControlFlowNode for List | provenance | Sink:MaD:3 Sink:MaD:3 | +| gemini_test.py:21:13:28:13 | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | gemini_test.py:20:18:29:9 | ControlFlowNode for List | provenance | Sink:MaD:3 Sink:MaD:3 Sink:MaD:3 | +| gemini_test.py:21:13:28:13 | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | gemini_test.py:20:18:29:9 | ControlFlowNode for List | provenance | Sink:MaD:3 Sink:MaD:3 Sink:MaD:3 Sink:MaD:3 | +| gemini_test.py:23:26:27:17 | ControlFlowNode for List [List element, Dictionary element at key text] | gemini_test.py:21:13:28:13 | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | provenance | | +| gemini_test.py:24:21:26:21 | ControlFlowNode for Dict [Dictionary element at key text] | gemini_test.py:23:26:27:17 | ControlFlowNode for List [List element, Dictionary element at key text] | provenance | | +| gemini_test.py:25:33:25:37 | ControlFlowNode for query | gemini_test.py:24:21:26:21 | ControlFlowNode for Dict [Dictionary element at key text] | provenance | | | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | langchain_test.py:3:26:3:32 | ControlFlowNode for request | provenance | | | langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:10:13:10:19 | ControlFlowNode for request | provenance | | -| langchain_test.py:10:5:10:9 | ControlFlowNode for query | langchain_test.py:21:28:21:51 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:2 | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:32:13:32:19 | ControlFlowNode for request | provenance | | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:47:13:47:19 | ControlFlowNode for request | provenance | | +| langchain_test.py:3:26:3:32 | ControlFlowNode for request | langchain_test.py:58:13:58:19 | ControlFlowNode for request | provenance | | +| langchain_test.py:10:5:10:9 | ControlFlowNode for query | langchain_test.py:17:34:17:38 | ControlFlowNode for query | provenance | Sink:MaD:20 | +| langchain_test.py:10:5:10:9 | ControlFlowNode for query | langchain_test.py:21:28:21:51 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:8 | | langchain_test.py:10:13:10:19 | ControlFlowNode for request | langchain_test.py:10:13:10:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | langchain_test.py:10:13:10:24 | ControlFlowNode for Attribute | langchain_test.py:10:13:10:37 | ControlFlowNode for Attribute() | provenance | dict.get | | langchain_test.py:10:13:10:37 | ControlFlowNode for Attribute() | langchain_test.py:10:5:10:9 | ControlFlowNode for query | provenance | | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | langchain_test.py:34:28:34:32 | ControlFlowNode for query | provenance | Sink:MaD:8 | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | langchain_test.py:35:27:35:31 | ControlFlowNode for query | provenance | Sink:MaD:8 | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | langchain_test.py:36:22:36:26 | ControlFlowNode for query | provenance | Sink:MaD:8 | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | langchain_test.py:37:39:37:43 | ControlFlowNode for query | provenance | Sink:MaD:8 | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | langchain_test.py:40:22:40:26 | ControlFlowNode for query | provenance | Sink:MaD:7 | +| langchain_test.py:32:13:32:19 | ControlFlowNode for request | langchain_test.py:32:13:32:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| langchain_test.py:32:13:32:24 | ControlFlowNode for Attribute | langchain_test.py:32:13:32:37 | ControlFlowNode for Attribute() | provenance | dict.get | +| langchain_test.py:32:13:32:37 | ControlFlowNode for Attribute() | langchain_test.py:32:5:32:9 | ControlFlowNode for query | provenance | | +| langchain_test.py:47:5:47:9 | ControlFlowNode for query | langchain_test.py:50:30:50:34 | ControlFlowNode for query | provenance | Sink:MaD:21 | +| langchain_test.py:47:13:47:19 | ControlFlowNode for request | langchain_test.py:47:13:47:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| langchain_test.py:47:13:47:24 | ControlFlowNode for Attribute | langchain_test.py:47:13:47:37 | ControlFlowNode for Attribute() | provenance | dict.get | +| langchain_test.py:47:13:47:37 | ControlFlowNode for Attribute() | langchain_test.py:47:5:47:9 | ControlFlowNode for query | provenance | | +| langchain_test.py:58:5:58:9 | ControlFlowNode for query | langchain_test.py:61:28:61:32 | ControlFlowNode for query | provenance | Sink:MaD:9 | +| langchain_test.py:58:5:58:9 | ControlFlowNode for query | langchain_test.py:62:15:62:19 | ControlFlowNode for query | provenance | Sink:MaD:10 | +| langchain_test.py:58:5:58:9 | ControlFlowNode for query | langchain_test.py:65:31:65:35 | ControlFlowNode for query | provenance | Sink:MaD:6 | +| langchain_test.py:58:5:58:9 | ControlFlowNode for query | langchain_test.py:68:60:68:64 | ControlFlowNode for query | provenance | Sink:MaD:5 | +| langchain_test.py:58:13:58:19 | ControlFlowNode for request | langchain_test.py:58:13:58:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | +| langchain_test.py:58:13:58:24 | ControlFlowNode for Attribute | langchain_test.py:58:13:58:37 | ControlFlowNode for Attribute() | provenance | dict.get | +| langchain_test.py:58:13:58:37 | ControlFlowNode for Attribute() | langchain_test.py:58:5:58:9 | ControlFlowNode for query | provenance | | | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:2:26:2:32 | ControlFlowNode for request | provenance | | | openai_test.py:2:26:2:32 | ControlFlowNode for request | openai_test.py:10:15:10:21 | ControlFlowNode for request | provenance | | | openai_test.py:2:26:2:32 | ControlFlowNode for request | openai_test.py:11:13:11:19 | ControlFlowNode for request | provenance | | @@ -51,41 +108,52 @@ edges | openai_test.py:10:15:10:21 | ControlFlowNode for request | openai_test.py:11:13:11:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | openai_test.py:10:15:10:26 | ControlFlowNode for Attribute | openai_test.py:10:15:10:41 | ControlFlowNode for Attribute() | provenance | dict.get | | openai_test.py:10:15:10:41 | ControlFlowNode for Attribute() | openai_test.py:10:5:10:11 | ControlFlowNode for persona | provenance | | -| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:16:15:16:19 | ControlFlowNode for query | provenance | Sink:MaD:5 | +| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:16:15:16:19 | ControlFlowNode for query | provenance | Sink:MaD:13 | | openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:27:28:27:32 | ControlFlowNode for query | provenance | | | openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:27:28:27:32 | ControlFlowNode for query | provenance | | | openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:40:28:40:32 | ControlFlowNode for query | provenance | | | openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:44:28:44:32 | ControlFlowNode for query | provenance | | -| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:51:16:51:36 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:3 | -| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:55:16:55:38 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:4 | -| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:60:16:60:36 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:6 | +| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:51:16:51:36 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:11 | +| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:55:16:55:38 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:12 | +| openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:60:16:60:36 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:14 | | openai_test.py:11:5:11:9 | ControlFlowNode for query | openai_test.py:66:17:66:43 | ControlFlowNode for BinaryExpr | provenance | | | openai_test.py:11:13:11:19 | ControlFlowNode for request | openai_test.py:11:13:11:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | openai_test.py:11:13:11:24 | ControlFlowNode for Attribute | openai_test.py:11:13:11:37 | ControlFlowNode for Attribute() | provenance | dict.get | | openai_test.py:11:13:11:37 | ControlFlowNode for Attribute() | openai_test.py:11:5:11:9 | ControlFlowNode for query | provenance | | -| openai_test.py:21:13:24:13 | ControlFlowNode for Dict [Dictionary element at key content] | openai_test.py:20:15:29:9 | ControlFlowNode for List | provenance | Sink:MaD:5 Sink:MaD:5 | +| openai_test.py:21:13:24:13 | ControlFlowNode for Dict [Dictionary element at key content] | openai_test.py:20:15:29:9 | ControlFlowNode for List | provenance | Sink:MaD:13 Sink:MaD:13 | | openai_test.py:23:28:23:51 | ControlFlowNode for BinaryExpr | openai_test.py:21:13:24:13 | ControlFlowNode for Dict [Dictionary element at key content] | provenance | | -| openai_test.py:25:13:28:13 | ControlFlowNode for Dict [Dictionary element at key content] | openai_test.py:20:15:29:9 | ControlFlowNode for List | provenance | Sink:MaD:5 Sink:MaD:5 | +| openai_test.py:25:13:28:13 | ControlFlowNode for Dict [Dictionary element at key content] | openai_test.py:20:15:29:9 | ControlFlowNode for List | provenance | Sink:MaD:13 Sink:MaD:13 | | openai_test.py:27:28:27:32 | ControlFlowNode for query | openai_test.py:25:13:28:13 | ControlFlowNode for Dict [Dictionary element at key content] | provenance | | | openrouter_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openrouter_test.py:2:26:2:32 | ControlFlowNode for request | provenance | | | openrouter_test.py:2:26:2:32 | ControlFlowNode for request | openrouter_test.py:10:13:10:19 | ControlFlowNode for request | provenance | | | openrouter_test.py:10:5:10:9 | ControlFlowNode for query | openrouter_test.py:21:28:21:32 | ControlFlowNode for query | provenance | | -| openrouter_test.py:10:5:10:9 | ControlFlowNode for query | openrouter_test.py:29:15:29:19 | ControlFlowNode for query | provenance | Sink:MaD:8 | -| openrouter_test.py:10:5:10:9 | ControlFlowNode for query | openrouter_test.py:34:15:34:19 | ControlFlowNode for query | provenance | Sink:MaD:7 | +| openrouter_test.py:10:5:10:9 | ControlFlowNode for query | openrouter_test.py:29:15:29:19 | ControlFlowNode for query | provenance | Sink:MaD:16 | +| openrouter_test.py:10:5:10:9 | ControlFlowNode for query | openrouter_test.py:34:15:34:19 | ControlFlowNode for query | provenance | Sink:MaD:15 | | openrouter_test.py:10:13:10:19 | ControlFlowNode for request | openrouter_test.py:10:13:10:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep | | openrouter_test.py:10:13:10:24 | ControlFlowNode for Attribute | openrouter_test.py:10:13:10:37 | ControlFlowNode for Attribute() | provenance | dict.get | | openrouter_test.py:10:13:10:37 | ControlFlowNode for Attribute() | openrouter_test.py:10:5:10:9 | ControlFlowNode for query | provenance | | models | 1 | Sink: Anthropic; Member[completions].Member[create].Argument[prompt:]; user-prompt-injection | -| 2 | Sink: LangChainChatModel; Member[invoke,stream,predict,call].Argument[0]; user-prompt-injection | -| 3 | Sink: OpenAI; Member[completions].Member[create].Argument[prompt:]; user-prompt-injection | -| 4 | Sink: OpenAI; Member[images].Member[generate,edit].Argument[prompt:]; user-prompt-injection | -| 5 | Sink: OpenAI; Member[responses].Member[create].Argument[input:]; user-prompt-injection | -| 6 | Sink: OpenAI; Member[videos].Member[create,create_and_poll,edit,remix,extend].Argument[prompt:]; user-prompt-injection | -| 7 | Sink: OpenRouter; Member[embeddings].Member[generate].Argument[input:]; user-prompt-injection | -| 8 | Sink: OpenRouter; Member[responses].Member[send].Argument[input:]; user-prompt-injection | -| 9 | Sink: agents; Member[Runner].Member[run,run_sync,run_streamed].Argument[1]; user-prompt-injection | -| 10 | Sink: agents; Member[Runner].Member[run,run_sync,run_streamed].Argument[input:]; user-prompt-injection | +| 2 | Sink: GoogleGenAI; Member[chats].Member[create].ReturnValue.Member[send_message,send_message_stream].Argument[0]; user-prompt-injection | +| 3 | Sink: GoogleGenAI; Member[models].Member[generate_content,generate_content_stream].Argument[contents:]; user-prompt-injection | +| 4 | Sink: GoogleGenAI; Member[models].Member[generate_images,generate_videos,edit_image].Argument[prompt:]; user-prompt-injection | +| 5 | Sink: LangChainAgent; Member[invoke,stream].Argument[0].DictionaryElement[messages].ListElement.DictionaryElement[content]; user-prompt-injection | +| 6 | Sink: LangChainAgentExecutor; Member[invoke].Argument[0].DictionaryElement[input]; user-prompt-injection | +| 7 | Sink: LangChainChatModel; Member[generate].Argument[0].ListElement.ListElement; user-prompt-injection | +| 8 | Sink: LangChainChatModel; Member[invoke,stream,predict,call].Argument[0]; user-prompt-injection | +| 9 | Sink: LangChainLLMChain; Member[invoke].Argument[0].DictionaryElement[input]; user-prompt-injection | +| 10 | Sink: LangChainLLMChain; Member[run].Argument[0]; user-prompt-injection | +| 11 | Sink: OpenAI; Member[completions].Member[create].Argument[prompt:]; user-prompt-injection | +| 12 | Sink: OpenAI; Member[images].Member[generate,edit].Argument[prompt:]; user-prompt-injection | +| 13 | Sink: OpenAI; Member[responses].Member[create].Argument[input:]; user-prompt-injection | +| 14 | Sink: OpenAI; Member[videos].Member[create,create_and_poll,edit,remix,extend].Argument[prompt:]; user-prompt-injection | +| 15 | Sink: OpenRouter; Member[embeddings].Member[generate].Argument[input:]; user-prompt-injection | +| 16 | Sink: OpenRouter; Member[responses].Member[send].Argument[input:]; user-prompt-injection | +| 17 | Sink: agents; Member[Runner].Member[run,run_sync,run_streamed].Argument[1]; user-prompt-injection | +| 18 | Sink: agents; Member[Runner].Member[run,run_sync,run_streamed].Argument[input:]; user-prompt-injection | +| 19 | Sink: google; Member[genai].Member[types].Member[CreateCachedContentConfig].Argument[contents:]; user-prompt-injection | +| 20 | Sink: langchain_core; Member[messages].Member[HumanMessage].Argument[content:]; user-prompt-injection | +| 21 | Sink: langchain_core; Member[prompts].Member[PromptTemplate].Instance.Member[format].Argument[any-named]; user-prompt-injection | nodes | agent_test.py:2:26:2:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | | agent_test.py:2:26:2:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | @@ -107,13 +175,52 @@ nodes | anthropic_test.py:11:13:11:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | | anthropic_test.py:20:28:20:32 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | | anthropic_test.py:29:16:29:55 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:3:26:3:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | +| gemini_test.py:3:26:3:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| gemini_test.py:11:5:11:9 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| gemini_test.py:11:13:11:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| gemini_test.py:11:13:11:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| gemini_test.py:11:13:11:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| gemini_test.py:15:18:15:22 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| gemini_test.py:20:18:29:9 | ControlFlowNode for List | semmle.label | ControlFlowNode for List | +| gemini_test.py:21:13:28:13 | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | semmle.label | ControlFlowNode for Dict [Dictionary element at key parts, List element, Dictionary element at key text] | +| gemini_test.py:23:26:27:17 | ControlFlowNode for List [List element, Dictionary element at key text] | semmle.label | ControlFlowNode for List [List element, Dictionary element at key text] | +| gemini_test.py:24:21:26:21 | ControlFlowNode for Dict [Dictionary element at key text] | semmle.label | ControlFlowNode for Dict [Dictionary element at key text] | +| gemini_test.py:25:33:25:37 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| gemini_test.py:25:33:25:37 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| gemini_test.py:33:35:33:58 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| gemini_test.py:37:16:37:20 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| gemini_test.py:43:22:43:26 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | | langchain_test.py:3:26:3:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | | langchain_test.py:3:26:3:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | | langchain_test.py:10:5:10:9 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | | langchain_test.py:10:13:10:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | | langchain_test.py:10:13:10:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | | langchain_test.py:10:13:10:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:17:34:17:38 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | | langchain_test.py:21:28:21:51 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| langchain_test.py:32:5:32:9 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:32:13:32:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:32:13:32:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| langchain_test.py:32:13:32:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:34:28:34:32 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:35:27:35:31 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:36:22:36:26 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:37:39:37:43 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:40:22:40:26 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:47:5:47:9 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:47:13:47:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:47:13:47:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| langchain_test.py:47:13:47:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:50:30:50:34 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:58:5:58:9 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:58:13:58:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | +| langchain_test.py:58:13:58:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| langchain_test.py:58:13:58:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| langchain_test.py:61:28:61:32 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:62:15:62:19 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:65:31:65:35 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | +| langchain_test.py:68:60:68:64 | ControlFlowNode for query | semmle.label | ControlFlowNode for query | | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember | | openai_test.py:2:26:2:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request | | openai_test.py:10:5:10:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona | @@ -149,11 +256,5 @@ nodes subpaths testFailures | agent_test.py:17:15:22:9 | ControlFlowNode for List | Unexpected result: Alert | -| gemini_test.py:3:35:3:44 | Comment # $ Source | Missing result: Source | -| gemini_test.py:15:26:15:60 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | -| gemini_test.py:25:40:25:74 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | -| gemini_test.py:33:62:33:96 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | -| gemini_test.py:37:24:37:58 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | -| gemini_test.py:43:30:43:64 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | -| langchain_test.py:17:43:17:77 | Comment # $ Alert[py/user-prompt-injection] | Missing result: Alert[py/user-prompt-injection] | +| gemini_test.py:20:18:29:9 | ControlFlowNode for List | Unexpected result: Alert | | openai_test.py:20:15:29:9 | ControlFlowNode for List | Unexpected result: Alert | diff --git a/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/langchain_test.py b/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/langchain_test.py index c57c65c7faa..b2d48847147 100644 --- a/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/langchain_test.py +++ b/python/ql/test/query-tests/Security/CWE-1427-UserPromptInjection/langchain_test.py @@ -20,3 +20,49 @@ def get_input_langchain(): result2 = model.invoke("Tell me about " + query) # $ Alert[py/user-prompt-injection] print(result1, result2) + + +@app.route("/langchain-providers") +def get_input_langchain_providers(): + from langchain_fireworks import ChatFireworks + from langchain_together import ChatTogether + from langchain_xai import ChatXAI + from langchain.chat_models import init_chat_model + + query = request.args.get("query") + + ChatFireworks().invoke(query) # $ Alert[py/user-prompt-injection] + ChatTogether().invoke(query) # $ Alert[py/user-prompt-injection] + ChatXAI().invoke(query) # $ Alert[py/user-prompt-injection] + init_chat_model("gpt-4.1").invoke(query) # $ Alert[py/user-prompt-injection] + + model = ChatOpenAI(model="gpt-4.1") + model.generate([[query]]) # $ Alert[py/user-prompt-injection] + + +@app.route("/langchain-prompts") +def get_input_langchain_prompts(): + from langchain_core.prompts import PromptTemplate + + query = request.args.get("query") + + template = PromptTemplate(template="Answer: {question}", input_variables=["question"]) + template.format(question=query) # $ Alert[py/user-prompt-injection] + + +@app.route("/langchain-chains") +def get_input_langchain_chains(): + from langchain.chains import LLMChain + from langchain.agents import AgentExecutor, create_agent + + query = request.args.get("query") + + chain = LLMChain() + chain.invoke({"input": query}) # $ Alert[py/user-prompt-injection] + chain.run(query) # $ Alert[py/user-prompt-injection] + + executor = AgentExecutor() + executor.invoke({"input": query}) # $ Alert[py/user-prompt-injection] + + agent = create_agent("gpt-4.1") + agent.invoke({"messages": [{"role": "user", "content": query}]}) # $ Alert[py/user-prompt-injection]