API Reference

Complete API documentation for InstantGrade.

Core Classes

Top-level instantgrade package exports.

Expose the high-level modules so users can import subpackages like:

from instantgrade.ingestion import file_loader

class instantgrade.Evaluator(solution_file_path, submission_folder_path, use_docker=True, parallel_workers=1, log_path='./logs', log_level='normal')[source]

Bases: object

The main orchestrator class responsible for grading workflows.

Parameters:

  • solution_file_path (str or Path) – Path to instructor’s reference solution notebook.

  • submission_folder_path (str or Path) – Folder containing all student notebooks.

  • use_docker (bool, optional) – Whether to use Docker-based isolated grading (default=True).

  • parallel_workers (int, optional) – Number of parallel workers for future extensions (default=1).

  • log_path (str, optional) – Path to directory for saving logs.

  • log_level (str, optional) – Logging verbosity (“debug”, “normal”, “silent”).

__init__(solution_file_path, submission_folder_path, use_docker=True, parallel_workers=1, log_path='./logs', log_level='normal')[source]
execute_all(submission_paths)[source]

Run grading across all students sequentially or in future parallel mode.

Return type:

List[Dict[str, Any]]

run()[source]

Run the full evaluation pipeline.

Return type:

ReportingService

summary(all_results)[source]

Generate quick text summary statistics from graded results.

Return type:

Dict[str, Any]

to_html(path)[source]

Generate HTML report for the graded results.

Evaluator

The main class for running evaluations.

class instantgrade.Evaluator(solution_file_path, submission_folder_path, use_docker=True, parallel_workers=1, log_path='./logs', log_level='normal')[source]

Bases: object

The main orchestrator class responsible for grading workflows.

Parameters:

  • solution_file_path (str or Path) – Path to instructor’s reference solution notebook.

  • submission_folder_path (str or Path) – Folder containing all student notebooks.

  • use_docker (bool, optional) – Whether to use Docker-based isolated grading (default=True).

  • parallel_workers (int, optional) – Number of parallel workers for future extensions (default=1).

  • log_path (str, optional) – Path to directory for saving logs.

  • log_level (str, optional) – Logging verbosity (“debug”, “normal”, “silent”).

__init__(solution_file_path, submission_folder_path, use_docker=True, parallel_workers=1, log_path='./logs', log_level='normal')[source]
run()[source]

Run the full evaluation pipeline.

Return type:

ReportingService

execute_all(submission_paths)[source]

Run grading across all students sequentially or in future parallel mode.

Return type:

List[Dict[str, Any]]

to_html(path)[source]

Generate HTML report for the graded results.

summary(all_results)[source]

Generate quick text summary statistics from graded results.

Return type:

Dict[str, Any]

Constructor

Evaluator(solution_path, submissions_dir, report_dir, **kwargs)

Parameters:

  • solution_path (str): Path to the reference solution file

  • submissions_dir (str): Directory containing student submissions

  • report_dir (str): Directory for output reports

  • timeout (int, optional): Execution timeout in seconds (default: 300)

  • allow_errors (bool, optional): Continue on errors (default: False)

  • compare_outputs (bool, optional): Compare cell outputs (default: True)

  • compare_variables (bool, optional): Compare variable states (default: True)

Returns:

  • Evaluator instance

Methods

run()

Execute the evaluation process.

results = evaluator.run()

Returns:

  • List of evaluation results

Ingestion Services

Services for loading solution and submission files.

Ingestion layer. Responsible for loading instructor and student submissions from various file formats (Jupyter, Excel, Python scripts, future languages).

SolutionIngestion

IngestionService

Execution Services

Services for executing notebooks and Excel files.

NotebookExecutor

Comparison Services

Services for comparing outputs and generating scores.

ComparisonService

Reporting Services

Services for generating HTML reports.

ReportingService

Utility Functions

Helper functions and utilities.

Utilities package for IO, logging and path helpers.

instantgrade.utils.generate_student_notebook(instructor_path, output_path)[source]

Create a student version of an instructor Jupyter notebook.

Generates a student notebook by: - Replacing all function bodies with ‘pass’ statements - Removing code cells containing assertions - Removing inline assert statements - Keeping all Markdown cells and overall structure

Parameters:

  • instructor_path (str or Path) – Path to the instructor’s notebook (.ipynb).

  • output_path (str or Path) – Destination path for the student version (.ipynb).

Raises:

FileNotFoundError – If the instructor notebook does not exist.

Examples

>>> generate_student_notebook(
...     "instructor_solution.ipynb",
...     "student_assignment.ipynb"
... )
✅ Student notebook generated at: student_assignment.ipynb

Notes

This function uses AST parsing to intelligently identify and stub out function definitions while preserving imports, assignments, and markdown content.

instantgrade.utils.remove_notebook_with_line(directory, line)[source]

Remove notebooks containing a specific line of code.

Searches for and deletes all Jupyter notebook files in the specified directory that contain a particular line of code in any code cell.

Parameters:

  • directory (str or Path) – The directory to search for Jupyter notebook files.

  • line (str) – The exact line of code to search for within notebooks.

Raises:

ValueError – If the specified path is not a directory.

Examples

>>> remove_notebook_with_line("submissions/", "assert solution == ")
Deleted notebook: submissions/test_notebook.ipynb

Warning

This function permanently deletes files. Use with caution.

instantgrade.utils.filename_safe_timestamp_format()[source]

Generate a filename-safe timestamp string.

Creates a timestamp string suitable for use in filenames, formatted as YYYYMMDD_HHMMSS.

Returns:

Timestamp string in the format ‘YYYYMMDD_HHMMSS’.

Return type:

str

Examples

>>> timestamp = filename_safe_timestamp_format()
>>> print(f"report_{timestamp}.html")
report_20251201_143022.html

Path Utilities

Path utilities for file and folder operations.

This module provides utility functions for working with file paths, detecting file types, and handling filesystem operations.

instantgrade.utils.path_utils.list_files_paths(path)[source]

List all files in the given directory.

Parameters:

path (Path) – Directory path to list files from.

Returns:

List of Path objects for all files in the directory. Returns empty list if path doesn’t exist or isn’t a directory.

Return type:

list of Path

Examples

>>> from pathlib import Path
>>> files = list_files_paths(Path("submissions/"))
>>> print(f"Found {len(files)} files")
instantgrade.utils.path_utils.is_notebook(path)[source]

Check if a file is a Jupyter notebook.

Parameters:

path (Path) – File path to check.

Returns:

True if the file has a .ipynb extension, False otherwise.

Return type:

bool

Examples

>>> from pathlib import Path
>>> is_notebook(Path("notebook.ipynb"))
True
>>> is_notebook(Path("script.py"))
False
instantgrade.utils.path_utils.is_excel(path)[source]

Check if a file is an Excel workbook.

Parameters:

path (Path) – File path to check.

Returns:

True if the file has a .xlsx or .xlsm extension, False otherwise.

Return type:

bool

Examples

>>> from pathlib import Path
>>> is_excel(Path("data.xlsx"))
True
>>> is_excel(Path("data.csv"))
False
instantgrade.utils.path_utils.get_file_extension(path)[source]

Get the lowercase file extension of a path.

Parameters:

path (Path) – File path to extract extension from.

Returns:

Lowercase file extension including the dot (e.g., ‘.ipynb’), or empty string if no extension or path has no suffix attribute.

Return type:

str

Examples

>>> from pathlib import Path
>>> get_file_extension(Path("notebook.ipynb"))
'.ipynb'
>>> get_file_extension(Path("README"))
''
instantgrade.utils.path_utils.filename_safe_timestamp_format()[source]

Generate a filename-safe timestamp string.

Creates a timestamp string suitable for use in filenames, formatted as YYYYMMDD_HHMMSS.

Returns:

Timestamp string in the format ‘YYYYMMDD_HHMMSS’.

Return type:

str

Examples

>>> timestamp = filename_safe_timestamp_format()
>>> print(f"report_{timestamp}.html")
report_20251201_143022.html

I/O Utilities

IO utilities for reading, writing, and managing files and folders.

This module provides utility functions for loading various file types, including Jupyter notebooks, Excel workbooks, JSON, CSV, and Python code files. It also includes notebook manipulation utilities.

instantgrade.utils.io_utils.safe_load_notebook(path)[source]

Safely load a Jupyter notebook and ensure cell IDs exist.

Reads a Jupyter notebook and automatically adds unique IDs to cells that are missing them. This prevents MissingIDFieldWarning in nbformat versions >=5.1.4. The file is not modified on disk.

Parameters:

path (Path) – Path to the Jupyter notebook file (.ipynb).

Returns:

The loaded and validated notebook object with all cells having IDs.

Return type:

nbformat.NotebookNode

Raises:

RuntimeError – If the notebook cannot be loaded or validated.

Examples

>>> from pathlib import Path
>>> nb = safe_load_notebook(Path("notebook.ipynb"))
>>> print(f"Loaded {len(nb.cells)} cells")

Notes

This function modifies the notebook structure in memory only. It does not write changes back to the file system.

instantgrade.utils.io_utils.normalize_notebook(path=None, inplace=True)[source]

Normalize a Jupyter notebook by ensuring unique cell IDs.

Ensures every cell in a Jupyter notebook has a unique ‘id’ field. This prevents MissingIDFieldWarning in nbformat >=5.1.4.

Parameters:

  • path (str or Path) – Path to the notebook file (.ipynb).

  • inplace (bool, default=True) – If True, overwrites the notebook file with normalized version. If False, returns the normalized object without modifying the file.

Returns:

The normalized notebook object with validated cell IDs.

Return type:

nbformat.NotebookNode

Examples

>>> from pathlib import Path
>>> nb = normalize_notebook("notebook.ipynb", inplace=False)
>>> print(f"Normalized {len(nb.cells)} cells")

See also

safe_load_notebook

Load notebook without modifying the file.

instantgrade.utils.io_utils.load_notebook(path)[source]

Load a Jupyter notebook file.

Parameters:

path (Path) – Path to the notebook file (.ipynb).

Returns:

The loaded notebook object.

Return type:

nbformat.NotebookNode

Examples

>>> from pathlib import Path
>>> nb = load_notebook(Path("notebook.ipynb"))
instantgrade.utils.io_utils.load_excel(path)[source]

Load an Excel workbook.

Parameters:

path (Path) – Path to the Excel file (.xlsx or .xlsm).

Returns:

The loaded Excel workbook object with formulas preserved.

Return type:

openpyxl.Workbook

Examples

>>> from pathlib import Path
>>> wb = load_excel(Path("data.xlsx"))
>>> print(wb.sheetnames)
instantgrade.utils.io_utils.load_json(path)[source]

Load a JSON file.

Parameters:

path (Path) – Path to the JSON file.

Returns:

The parsed JSON data as a dictionary.

Return type:

dict

Examples

>>> from pathlib import Path
>>> data = load_json(Path("config.json"))
instantgrade.utils.io_utils.load_csv(path)[source]

Load a CSV file into a pandas DataFrame.

Parameters:

path (Path) – Path to the CSV file.

Returns:

The loaded CSV data.

Return type:

pandas.DataFrame

Examples

>>> from pathlib import Path
>>> df = load_csv(Path("data.csv"))
>>> print(df.head())
instantgrade.utils.io_utils.load_raw_code(path)[source]

Load raw Python code from a file.

Parameters:

path (Path) – Path to the Python file (.py).

Returns:

The raw Python code as a string.

Return type:

str

Examples

>>> from pathlib import Path
>>> code = load_raw_code(Path("script.py"))
>>> print(len(code))
instantgrade.utils.io_utils.generate_student_notebook(instructor_path, output_path)[source]

Create a student version of an instructor Jupyter notebook.

Generates a student notebook by: - Replacing all function bodies with ‘pass’ statements - Removing code cells containing assertions - Removing inline assert statements - Keeping all Markdown cells and overall structure

Parameters:

  • instructor_path (str or Path) – Path to the instructor’s notebook (.ipynb).

  • output_path (str or Path) – Destination path for the student version (.ipynb).

Raises:

FileNotFoundError – If the instructor notebook does not exist.

Examples

>>> generate_student_notebook(
...     "instructor_solution.ipynb",
...     "student_assignment.ipynb"
... )
✅ Student notebook generated at: student_assignment.ipynb

Notes

This function uses AST parsing to intelligently identify and stub out function definitions while preserving imports, assignments, and markdown content.

instantgrade.utils.io_utils.remove_notebook_with_line(directory, line)[source]

Remove notebooks containing a specific line of code.

Searches for and deletes all Jupyter notebook files in the specified directory that contain a particular line of code in any code cell.

Parameters:

  • directory (str or Path) – The directory to search for Jupyter notebook files.

  • line (str) – The exact line of code to search for within notebooks.

Raises:

ValueError – If the specified path is not a directory.

Examples

>>> remove_notebook_with_line("submissions/", "assert solution == ")
Deleted notebook: submissions/test_notebook.ipynb

Warning

This function permanently deletes files. Use with caution.

Command-Line Interface

The CLI module for terminal usage.

Command-line interface for InstantGrade.

This module provides CLI commands for running evaluations from the terminal.

Data Models

EvaluationResult

Result object returned by the evaluator.

Attributes:

  • filename (str): Name of the evaluated submission

  • score (float): Score out of 100

  • status (str): Evaluation status (‘PASSED’, ‘FAILED’, ‘ERROR’)

  • report_path (str): Path to the generated HTML report

  • execution_time (float): Time taken to execute (seconds)

  • error_message (str, optional): Error message if evaluation failed

Example:

results = evaluator.run()
for result in results:
    print(f"{result.filename}: {result.score}/100")
    print(f"Status: {result.status}")
    print(f"Report: {result.report_path}")

Exceptions

EvaluationError

Raised when evaluation fails.

class EvaluationError(Exception):
    """Raised when evaluation process fails."""
    pass

ExecutionError

Raised when notebook execution fails.

class ExecutionError(Exception):
    """Raised when notebook execution fails."""
    pass

IngestionError

Raised when file ingestion fails.

class IngestionError(Exception):
    """Raised when file loading fails."""
    pass

Constants

# Default timeout for notebook execution (seconds)
DEFAULT_TIMEOUT = 300

# Supported file extensions
SUPPORTED_NOTEBOOK_EXTENSIONS = ['.ipynb']
SUPPORTED_EXCEL_EXTENSIONS = ['.xlsx', '.xls']

# Report template
DEFAULT_REPORT_TEMPLATE = 'default.html'

Type Hints

InstantGrade uses type hints throughout the codebase:

from typing import List, Dict, Optional, Union
from pathlib import Path

def evaluate_submission(
    solution_path: Union[str, Path],
    submission_path: Union[str, Path],
    timeout: Optional[int] = None
) -> Dict[str, any]:
    """Evaluate a single submission."""
    pass

Usage Examples

Basic Usage

from instantgrade import Evaluator

evaluator = Evaluator(
    solution_path="solution.ipynb",
    submissions_dir="submissions/",
    report_dir="reports/"
)

results = evaluator.run()

With Type Hints

from instantgrade import Evaluator
from typing import List

def grade_assignments(
    solution: str,
    submissions: str,
    output: str
) -> List[dict]:
    evaluator: Evaluator = Evaluator(
        solution_path=solution,
        submissions_dir=submissions,
        report_dir=output
    )
    return evaluator.run()

Error Handling

from instantgrade import Evaluator, EvaluationError

try:
    evaluator = Evaluator(
        solution_path="solution.ipynb",
        submissions_dir="submissions/",
        report_dir="reports/"
    )
    results = evaluator.run()
except EvaluationError as e:
    print(f"Evaluation failed: {e}")

Advanced API Usage

Custom Comparison

from instantgrade import Evaluator
from instantgrade.comparison import ComparisonService

class CustomComparison(ComparisonService):
    def compare_outputs(self, expected, actual):
        # Custom logic here
        return super().compare_outputs(expected, actual)

evaluator = Evaluator(
    solution_path="solution.ipynb",
    submissions_dir="submissions/",
    report_dir="reports/",
    comparison_service=CustomComparison()
)

Custom Reporting

from instantgrade import Evaluator
from instantgrade.reporting import ReportingService

class CustomReporting(ReportingService):
    def generate_report(self, comparison_result):
        # Custom report generation
        pass

evaluator = Evaluator(
    solution_path="solution.ipynb",
    submissions_dir="submissions/",
    report_dir="reports/",
    reporting_service=CustomReporting()
)

See Also