Expressions
Refiner's expression DSL for vectorized transforms
Expressions describe columnar computations for vectorized transforms.
Constructors
mdr.col("episode_id")
mdr.lit("train")Boolean logic
Use &, |, and ~. Do not use Python and or or.
pipeline = pipeline.filter(
(mdr.col("num_frames") > 30) & (mdr.col("split") == "train")
)Conditional values
pipeline = pipeline.with_columns(
bucket=mdr.if_else(mdr.col("num_frames") > 300, "long", "short")
)Null handling
pipeline = pipeline.with_columns(
task=mdr.coalesce(mdr.col("task"), mdr.lit("unknown"))
)String and datetime helpers
Expression objects include namespaces for common string and datetime operations when supported by the underlying Arrow execution.
pipeline = pipeline.filter(mdr.col("episode_id").str.contains("pick"))