The protocols

The API presented by this plugin is defined under pytest_typing_runner.protocols

protocol pytest_typing_runner.protocols.RunCleaner

Callable used to perform some cleanup action

Classes that implement this protocol must have the following methods / attributes:

__call__() → None: Call self as a function.

protocol pytest_typing_runner.protocols.RunCleaners

A collection of RunCleaner objects

Classes that implement this protocol must have the following methods / attributes:

add(unique_identifier: str, cleaner: RunCleaner, /) → None

Register a cleaner.

If a cleaner with this identifier has already been registered then it will be overridden

__iter__() → Iterator[RunCleaner]: Yield all the unique cleaners

class pytest_typing_runner.protocols.RunOptionsCloneArgs: Used to represent the options that can be changed when cloning run options

protocol pytest_typing_runner.protocols.RunOptions

Used to represent the options used to run a type checker

Classes that implement this protocol must have the following methods / attributes:

property scenario_runner: ScenarioRunner[T_Scenario]: The scenario runner

property cwd: Path: The working directory to run the type checker from

property program_runner_maker: ProgramRunnerMaker[T_Scenario]: Make the specific type checker to run

property args: Sequence[str]: The arguments to pass into the type checker

property check_paths: Sequence[str]: The paths to check with the type checker

property do_followup: bool: Return whether followup runs should be done

property environment_overrides: Mapping[str, str | None]: Overrides of environment variables for when running the type checker

clone(*, program_runner_maker: ProgramRunnerMaker[T_Scenario] | None = None, **kwargs: Unpack[RunOptionsCloneArgs]) → Self: Used to provide a copy with certain options changed

protocol pytest_typing_runner.protocols.RunOptionsModify

Represents a function that can return a modified run options

Classes that implement this protocol must have the following methods / attributes:

__call__(options: RunOptions[T_Scenario], /) → RunOptions[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.FileModifier

Represents a function that can change a file in the scenario

Implementations should aim to consider the signature as follows:

Parameters:

path – A string representing the path from the root dir to a file
content – Passed in as None if the file is to be deleted, otherwise the content to override the file with

Classes that implement this protocol must have the following methods / attributes:

__call__(*, path: str, content: str | None) → None: Call self as a function.

protocol pytest_typing_runner.protocols.NoticeChecker

Used to represent a function that can be wrapped to provide a function for checking notices

Classes that implement this protocol must have the following methods / attributes:

property result: RunResult: The result to check against

property run_options: RunOptions[T_Scenario]: The final options used for the run

property runner: ProgramRunner[T_Scenario]: The runner that was used to make the result

check(expected_notices: ProgramNotices, /) → None

protocol pytest_typing_runner.protocols.RunResult

Used to represent the result of running a type checker

Classes that implement this protocol must have the following methods / attributes:

property exit_code: int: The exit code from running the type checker

property stdout: str: The stdout from running the type checker

property stderr: str: The stderr from running the type checker

protocol pytest_typing_runner.protocols.RunnerConfig

An object to represent all the options relevant to this pytest plugin

A default implementation is provided by pytest_typing_runner.RunnerConfig

Classes that implement this protocol must have the following methods / attributes:

property same_process: bool

Set by the –same-process option.

Used to know if the type checker should be run in the same process or not.

property typing_strategy: Strategy

Set by the –typing-strategy option.

Used to know what type checker should be used and how.

protocol pytest_typing_runner.protocols.ProgramRunner

Used to run the static type checker

Classes that implement this protocol must have the following methods / attributes:

property options: RunOptions[T_Scenario]: The options for this run

run() → NoticeChecker[T_Scenario]: Run the static type checker and return a result

short_display() → str: Return a string to represent the command that was run

protocol pytest_typing_runner.protocols.ProgramRunnerMaker

Used to contruct a program runner

Classes that implement this protocol must have the following methods / attributes:

property default_args: Sequence[str]: The default args to start the program runner with

property do_followups: bool: The default for whether the program runner should be used in a followup run

property same_process: bool: Whether this program runner executes within this process instead of in a subprocess

property is_daemon: bool: Whether this program runner uses a daemon

property program_short: str: String representing the family of type checker (i.e. “mypy”, “pyright”, etc)

__call__(*, options: RunOptions[T_Scenario]) → ProgramRunner[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.ProgramRunnerChooser

Used to choose which program runner to use

Classes that implement this protocol must have the following methods / attributes:

__call__(*, config: RunnerConfig, scenario: T_Scenario) → ProgramRunnerMaker[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.Strategy

typing.Protocol.

Classes that implement this protocol must have the following methods / attributes:

property program_short: str: String representing the family of type checker (i.e. “mypy”, “pyright”, etc)

property program_runner_chooser: ProgramRunnerChooser: Used to make a program runner

protocol pytest_typing_runner.protocols.StrategyMaker

Used to construct an object implementing Strategy

Classes that implement this protocol must have the following methods / attributes:

__call__() → Strategy: Call self as a function.

protocol pytest_typing_runner.protocols.StrategyRegistry

Used to register different typing strategies

Classes that implement this protocol must have the following methods / attributes:

register(*, name: str, description: str, maker: StrategyMaker, make_default: bool = False) → None: Register a maker to a specific name

remove_strategy(*, name: str) → None: Remove a strategy

set_default(*, name: str) → None: Set the default strategy

get_strategy(*, name: str) → tuple[str, StrategyMaker] | None: Return the strategy for the provided name if one exists

property default: str: Return the default strategy

property choices: list[str]: Return the known strategies

protocol pytest_typing_runner.protocols.StrategyRegisterer

Used to register strategies

Classes that implement this protocol must have the following methods / attributes:

__call__(registry: StrategyRegistry, /) → None: Call self as a function.

protocol pytest_typing_runner.protocols.ScenarioRun

Used to hold information about a single run of a type checker

Classes that implement this protocol must have the following methods / attributes:

property is_first: bool: Whether this is the first run for this scenario

property is_followup: bool: Whether this is a followup run

property file_modifications: Sequence[tuple[str, str]]: The file modifications that were done before this run

property checker: NoticeChecker[T_Scenario]: The result from running the type checker

property expectation_error: Exception | None: Any error from matching the result to the expectations for that run

for_report() → Iterator[str]: Used to yield strings returned to present in the pytest report

protocol pytest_typing_runner.protocols.ScenarioRuns

Represents information to return in a pytest report at the end of the test

A default implementation is provided by pytest_typing_runner.ScenarioRuns

Classes that implement this protocol must have the following methods / attributes:

property has_runs: bool: Whether there were any runs to report

property known_files: Set[str]: The paths to all the files created as part of the scenario

property scenario: T_Scenario: The scenario these runs belong to

for_report() → Iterator[str]: Used to yield strings to place into the pytest report

add_file_modification(path: str, action: str) → None: Used to record a file modification for the next run

add_run(*, checker: NoticeChecker[T_Scenario], expectation_error: Exception | None) → ScenarioRun[T_Scenario]: Used to add a single run to the record

protocol pytest_typing_runner.protocols.ScenarioRunsMaker

Used to construct a scenario runs

Classes that implement this protocol must have the following methods / attributes:

__call__(*, scenario: T_Scenario) → ScenarioRuns[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.Severity

Used to represent the severity of a notice

Classes that implement this protocol must have the following methods / attributes:

property display: str: Return the severity as a string

__eq__(other: object) → bool: Determine if this is equal to another object

__lt__(other: Severity) → bool: To allow ordering a sequence of severity objects

__hash__ = None

protocol pytest_typing_runner.protocols.ProgramNoticesChanger

typing.Protocol.

Classes that implement this protocol must have the following methods / attributes:

__call__(notices: ProgramNotices, /) → ProgramNotices: Call self as a function.

protocol pytest_typing_runner.protocols.FileNoticesChanger

typing.Protocol.

Classes that implement this protocol must have the following methods / attributes:

__call__(notices: FileNotices, /) → FileNotices: Call self as a function.

protocol pytest_typing_runner.protocols.LineNoticesChanger

typing.Protocol.

Classes that implement this protocol must have the following methods / attributes:

__call__(notices: LineNotices, /) → LineNotices | None: Call self as a function.

protocol pytest_typing_runner.protocols.ProgramNoticeChanger

typing.Protocol.

Classes that implement this protocol must have the following methods / attributes:

__call__(notice: ProgramNotice, /) → ProgramNotice | None: Call self as a function.

protocol pytest_typing_runner.protocols.NoticeMsg

Used to represent the message on a notice and the ability to compare with other messages

Classes that implement this protocol must have the following methods / attributes:

raw: str

__str__() → str: Return str(self).

__eq__(o: object, /) → bool: Return self==value.

__hash__() → int: Return hash(self).

__lt__(o: NoticeMsg, /) → bool: Return self<value.

property is_plain: bool: Whether this msg does no pattern matching

replace(old: str, new: str, count: int = -1) → Self: A shortcut for msg.clone(raw=msg.raw.replace(old, new, count))

match(*, want: str) → bool

Parameters:: want – The string to match against
Returns:: True if want matches some expectation

clone(*, pattern: str) → Self: Used to create a version of this msg class but for a different msg

split_lines() → Iterator[Self]: Yield copies of this notice with a clone per line

protocol pytest_typing_runner.protocols.NoticeMsgMaker

Used to create a NoticeMsg

Classes that implement this protocol must have the following methods / attributes:

__call__(*, pattern: str) → NoticeMsg: Call self as a function.

protocol pytest_typing_runner.protocols.NoticeMsgMakerMap

Holds notice msg makers

Classes that implement this protocol must have the following methods / attributes:

get(name: str, /, default: NoticeMsgMaker) → NoticeMsgMaker
get(name: str, /, default: None = None) → NoticeMsgMaker | None

property available: Sequence[str]

class pytest_typing_runner.protocols.ProgramNoticeCloneKwargs

class pytest_typing_runner.protocols.ProgramNoticeCloneAndMsgKwargs

protocol pytest_typing_runner.protocols.ProgramNotice

Represents a single notice from the static type checker

Classes that implement this protocol must have the following methods / attributes:

property location: Path: The file this notice is contained in

property line_number: int: The line number this notice appears on

property col: int | None: The column this notice is found on, if one is provided

property severity: Severity: The severity of the notice

property msg: NoticeMsg: The message attached to the notice, dedented and including newlines

property is_type_reveal: bool: Returns whether this notice represents output from a reveal_type(…) instruction

clone(*, msg: str | NoticeMsg | None = None, **kwargs: Unpack[ProgramNoticeCloneKwargs]) → Self: Return a clone with specific changes

__lt__(other: ProgramNotice) → bool: Make Program notices Orderable

matches(other: ProgramNotice) → bool: Return whether this matches the provided notice

display() → str: Return a string form for display

protocol pytest_typing_runner.protocols.DiffFileNotices

Represents the left/right of a diff between notices for a file

Classes that implement this protocol must have the following methods / attributes:

__iter__() → Iterator[tuple[int, Sequence[ProgramNotice], Sequence[ProgramNotice]]]

protocol pytest_typing_runner.protocols.DiffNotices

Represents the difference between two ProgramNotices per file

Classes that implement this protocol must have the following methods / attributes:

__iter__() → Iterator[tuple[str, DiffFileNotices]]

protocol pytest_typing_runner.protocols.LineNotices

Represents the information returned by the static type checker for a specific line in a file

Classes that implement this protocol must have the following methods / attributes:

property line_number: int: The line number these notices are for

property location: Path: The path to this file as represented by the type checker

property has_notices: bool: Whether this has any notices

property msg_maker: NoticeMsgMaker: Implementations should use this when generating a program notice

__iter__() → Iterator[ProgramNotice]: Yield all the notices

set_notices(notices: Sequence[ProgramNotice | None], *, allow_empty: Literal[True]) → Self

set_notices(notices: Sequence[ProgramNotice | None], *, allow_empty: Literal[False] = False) → Self | None

Return a copy where the chosen notice(s) are replaced

Parameters:

notices – The notices the clone should have. Any None entries are dropped
allow_empty – If False then None is returned instead of a copy with an empty list

generate_notice(*, msg: str | NoticeMsg, msg_maker: NoticeMsgMaker | None = None, severity: Severity | None = None, col: int | None = None) → ProgramNotice

Generate a notice for this location and line

This does not add the notice to this LineNotices

protocol pytest_typing_runner.protocols.FileNotices

Represents the information returned by the static type checker for a specific file

Classes that implement this protocol must have the following methods / attributes:

property location: Path: The path to this file as represented by the type checker

property has_notices: bool: Whether this file has notices

property known_names: Mapping[str, int]: Return the registered names

known_line_numbers() → Iterator[int]: Yield the line numbers that have line notices

property msg_maker: NoticeMsgMaker: Implementations should pass this on when generating a line notices

__iter__() → Iterator[ProgramNotice]: Yield all the notices

get_line_number(name_or_line: str | int, /) → int | None: Given a name or line number, return a line number or None if that line number doesn’t have any notices

notices_at_line(line_number: int) → LineNotices | None: Return the line notices for a specific line number if there are any

generate_notices_for_line(line_number: int, *, msg_maker: NoticeMsgMaker | None = None) → LineNotices

Return a line notices for this location at the specified line

Implementations should not add this generated object to itself.

set_name(name: str, line_number: int) → Self: Associate a name with a specific line number

set_lines(notices: Mapping[int, LineNotices | None]) → Self

Return a modified notices with these notices for the specified line numbers

Any None values will result in that line number being removed

clear(*, clear_names: bool) → Self

Return a modified file notices with all notices removed

Parameters:: clear_names – Whether to clear names as well

protocol pytest_typing_runner.protocols.FileNoticesParser

Used to parse notices from comments in a file

Classes that implement this protocol must have the following methods / attributes:

__call__(content: str, /, *, into: FileNotices) → tuple[str, FileNotices]: Call self as a function.

protocol pytest_typing_runner.protocols.ProgramNotices

Represents the information returned by the static type checker

Classes that implement this protocol must have the following methods / attributes:

property has_notices: bool: Whether there were any notices

__iter__() → Iterator[ProgramNotice]: Yield all the notices

property msg_maker: NoticeMsgMaker: Implementations should pass this on when generating a file notices

known_locations() → Iterator[Path]: Yield locations that have associated file notices

diff(root_dir: Path, other: ProgramNotices) → DiffNotices: Return an object representing what is the same and what is different between two program notices

notices_at_location(location: Path) → FileNotices | None: Return the notices for this location if any

set_files(notices: Mapping[Path, FileNotices | None]) → Self: Return a copy with these notices for the specified files

generate_notices_for_location(location: Path, *, msg_maker: NoticeMsgMaker | None = None) → FileNotices

Return a file notices for this location

Implementations should not modify this ProgramNotices

protocol pytest_typing_runner.protocols.Expectations

This objects knows what to expect from running the static type checker

Classes that implement this protocol must have the following methods / attributes:

check(*, notice_checker: NoticeChecker[T_Scenario]) → None: Used to check the result against these expectations

protocol pytest_typing_runner.protocols.ExpectationsMaker

Callable that creates an Expectations object

Classes that implement this protocol must have the following methods / attributes:

__call__() → Expectations[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.ExpectationsSetup

Used to setup an expectations maker

Classes that implement this protocol must have the following methods / attributes:

__call__(*, options: RunOptions[T_Scenario]) → ExpectationsMaker[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.Expects

Holds boolean expectations

Classes that implement this protocol must have the following methods / attributes:

failure: bool: Whether the assertions in the test are expected to fail

daemon_restarted: bool: Whether a daemon restart is expected

protocol pytest_typing_runner.protocols.Scenario

Used to hold relevant information for running and testing a type checker run.

This object is overridden to provide a mechanism for stringing custom data throughout all the other objects.

A default implementation is provided by pytest_typing_runner.Scenario

The typing_scenario_maker fixture can be defined to return the exact concrete implementation to use for a particular scope.

Classes that implement this protocol must have the following methods / attributes:

property expects: Expects: Boolean expectations

property root_dir: Path: The root directory for all files in the scenario

protocol pytest_typing_runner.protocols.ScenarioRunner

Used to facilitate the running and testing of a type checker run.

A default implementation is provided by pytest_typing_runner.ScenarioRunner

The typing_ fixture can be defined to return the exact concrete implementation to use for a particular scope.

Classes that implement this protocol must have the following methods / attributes:

property scenario: T_Scenario: The scenario under test

property default_program_runner_maker: ProgramRunnerMaker[T_Scenario]: Default constructor for making a program runner

property cleaners: RunCleaners: An object to register cleanup functions for the end of the run

execute_static_checking(*, options: RunOptions[T_Scenario]) → NoticeChecker[T_Scenario]: Called to use the run options to run a type checker and get a result

run_and_check(*, options: RunOptions[T_Scenario], setup_expectations: ExpectationsSetup[T_Scenario]) → None: Used to do a run of a type checker and check against the provided expectations

property runs: ScenarioRuns[T_Scenario]: The runs of the type checker for this scenario

prepare_scenario() → None: Called when the scenario has been created. This method may do any mutations it wants on self.scenario

cleanup_scenario() → None: Called after the test is complete. This method may do anything it wants for cleanup

add_to_pytest_report(name: str, sections: list[tuple[str, str]]) → None: Used to add a section to the pytest report

file_modification(path: str, content: str | None) → None: Used to modify a file for the scenario and record it on the runs

normalise_program_runner_notice(options: RunOptions[T_Scenario], notice: ProgramNotice, /) → ProgramNotice: Used to normalise each notice parsed from the output of the program runner

generate_program_notices(*, msg_maker: NoticeMsgMaker | None = None) → ProgramNotices: Return an object that satisfies an empty ProgramNotices

protocol pytest_typing_runner.protocols.ScenarioMaker

Represents a callable that creates Scenario objects

Classes that implement this protocol must have the following methods / attributes:

__call__(*, config: RunnerConfig, root_dir: Path) → T_CO_Scenario: Call self as a function.

protocol pytest_typing_runner.protocols.ScenarioRunnerMaker

Represents an object that creates Scenario Runner objects

Classes that implement this protocol must have the following methods / attributes:

__call__(*, config: RunnerConfig, root_dir: Path, scenario_maker: ScenarioMaker[T_Scenario], scenario_runs_maker: ScenarioRunsMaker[T_Scenario]) → ScenarioRunner[T_Scenario]: Call self as a function.

protocol pytest_typing_runner.protocols.ScenarioFile

Used to hold information about a file in a scenario

Classes that implement this protocol must have the following methods / attributes:

property root_dir: Path: The root dir of the scenario

property path: str: The path to this file relative to the rootdir

notices(*, into: FileNotices) → FileNotices | None: Return the notices associated with this file

protocol pytest_typing_runner.protocols.ScenarioFileMaker

Callable that returns a ScenarioFile

Classes that implement this protocol must have the following methods / attributes:

__call__(*, root_dir: Path, path: str) → T_CO_ScenarioFile: Call self as a function.