context-leak good first issues (Python)

Add a --fail-on-violation flag so CI can gate on any disclosure 3 months ago

good first issue mentored type: feature area: infra size: S

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Validate that a scenario's forbidden and appropriate_flows do not overlap 3 months ago

good first issue type: feature area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Write a scenario-authoring guide (choosing values, aliases, forbidden vs appropriate) 3 months ago

good first issue type: docs area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add tests pinning the empty-forbidden and empty-appropriate rate conventions 3 months ago

good first issue mentored type: test area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add --show-matrix to print a scenario's appropriateness matrix 3 months ago

good first issue type: feature area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a joint 'clean-and-complete' rate to the aggregate report 3 months ago

help wanted type: feature area: infra size: M

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a synthetic clinical scenario using a non-monetary meaningful-but-unique value 3 months ago

help wanted type: feature area: privacy size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a scenario where an attribute is appropriate to the data subject but forbidden to a third party 3 months ago

good first issue mentored type: feature area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Detect confirm/deny disclosures in the deterministic value match 3 months ago

help wanted type: research area: agent-safety size: M

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Validate that a scenario's attribute names and recipient ids are unique 3 months ago

good first issue mentored type: feature area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add unit tests for score()'s forbidden×output crossing and rate math 3 months ago

good first issue mentored type: test area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Print violations as readable `attribute → recipient (role)` lines in the CLI 3 months ago

good first issue type: feature area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add unit tests for the scorer's normalization and alias matching 3 months ago

good first issue mentored type: test area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a `--json` output mode to the CLI 3 months ago

good first issue mentored type: feature area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Pin a non-degenerate bootstrap CI in a test 3 months ago

good first issue

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add `--list-scenarios` and `--scenario <id>` to the CLI 3 months ago

good first issue type: feature area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Validate that a scenario's forbidden/appropriate_flows reference real ids 3 months ago

good first issue type: feature area: infra size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a second synthetic scenario in a different neutral domain 3 months ago

good first issue type: feature area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add scorer normalization edge-case tests (NFKC / case-fold / whitespace) 3 months ago

good first issue type: test area: agent-safety size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a disclosure-rate + utility report across scenarios 3 months ago

help wanted type: feature area: infra size: M

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

Add a multi-attribute scenario where different attributes have different forbidden recipients 3 months ago

good first issue type: feature area: llm size: S

bamdadd/context-leak

0

Leakage-verified contextual-integrity benchmark: does an agent disclose a sensitive attribute to a recipient the context forbids. Defensive, synthetic-only. Sibling of leakgauge.

Python

#agent-safety#benchmark#contextual-integrity#evaluation#llm#open-source#privacy

bamdadd/context-leak

Open Issues Need Help