Skip to content

read_file_in_chunks fails when pandas not available #344

Description

@f-hafner

Bug

Reading a file in chunks fails with output format polars when pandas is not in the virtual environment.

To reproduce

# script.py
import polars as pl
import pyreadstat
import tempfile

if __name__ == "__main__":

    with tempfile.NamedTemporaryFile(suffix=".sav") as f:
        data_dict = {
            "a": [1,2,3,4],
            "b": [10,11,12,13]
        }
        data = pl.DataFrame(data_dict)
        pyreadstat.write_sav(data, f.name)

        for chunk, _ in pyreadstat.read_file_in_chunks(
               read_function=pyreadstat.read_sav,
               chunksize=2,
               file_path=f,
               output_format="polars",
               ):
            assert isinstance(chunk, pl.DataFrame)
            print("Success.")

Run with

uv run --with "polars,pandas,pyreadstat" script.py # works
uv run --with "polars,pyreadstat" script.py # fails

Error message:

Traceback (most recent call last):
  File "pyreadstat/_readstat_parser.pyx", line 1276, in pyreadstat._readstat_parser.run_conversion
    import pandas
ModuleNotFoundError: No module named 'pandas'
[...]
pyreadstat._readstat_parser.PyreadstatError: You requested pandas as output_format but cannot import pandas

File example

see above

Expected behavior

No error when pandas is not in the venv.

Setup Information:

How did you install pyreadstat? uv (pip, conda, directly from repo)
Platform (windows, macOS, linux, 32 or 64 bit) linux 64bit
Python Version 3.14.4
Python Distribution (System, plain python, Anaconda): via uv
Using Virtualenv or condaenv? with inline dependencies via uv

Solution

I think the the output_format argument is not passed when reading the metadata here?

_, meta = read_function(file_path, metadataonly=True)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions