Polars

@pola

Dataframes powered by a multithreaded, vectorized query engine, written in Rust.

3 Following1,046 Followers86 Posts

Bluesky

Is this your profile?Claim it and pola.dirty.social is yours.Claim profile

Polars@pola· 6h

Time series datasets are seldom perfect. Polars gives you a whole menu for repairing them, and the right pick depends on what you need.

Polars@pola· 14d

Aggregating 500GB of compressed parquet with 16,000,000,000 rows to display it in an enterprise-ready Plotly dashboard. Check the dashboard: polars-cloud-demo.plotly.app And the blog: pola.rs/posts/market...

Polars@pola· 19d

We've released Python Polars 1.43. Some of the highlights: • pl.list() • ewm_sum() and ewm_sum_by() • Faster joins on hive-partitioned data Blog post: pola.rs/posts/polars... Full changelog: github.com/pola-rs/pola...

Polars@pola· 26d

A float column can hold two different kinds of missing: null for a value that is absent, and NaN for arithmetic that had no valid answer (think 0.0/0.0). Polars keeps them strictly separate.

Polars@pola· 28d

Reading a terabyte dataset from S3 goes fastest on a cluster of small machines, while heavy joins run fastest on one big machine. We benchmarked single node Polars against distributed Polars on the same total resources, and the bottleneck of your query decides the winner. pola.rs/posts/single...

Polars@pola· Jul 7

We've been busy in Q2 2026. Read all the highlights in the latest Polars in Aggregate: pola.rs/posts/polars...

Polars@pola· Jun 29

We've released Python Polars 1.42. Some of the highlights: • Adaptive cloud I/O concurrency - up to 4x faster reads • Contradictory filter elimination • is_sorted() for DataFrame and Expressions Blog post: pola.rs/posts/polars... Full changelog: github.com/pola-rs/pola...

Polars@pola· Jun 25

How well do LLMs migrate pandas to Polars by themselves? We tested how well Claude translates a pandas corpus to Polars. Results were promising but not perfect. To improve this, we built a Polars skill that helps the agent. Read the full post here: pola.rs/posts/llm-po...

Polars@pola· Jun 16

Distributed Polars is 3x faster than Spark on the PDSH benchmark and up to 7.8x faster on individual queries. Read the full benchmark post here: https:/pola.rs/posts/polars-pyspark-benchmarks/

Figure showing the speed-up of Polars compared to PySpark per query of the PDS-H benchmark.

Polars@pola· Jun 3

Run Polars' distributed engine on your own infrastructure. Deploy a distributed Polars cluster on any Kubernetes setup (EKS, AKS, GKE, or minikube) and get a query dashboard with past queries, advanced query profiling, Open-lineage support, and more. More at pola.rs/posts/polars...

Polars@pola· May 14

Polars supports a full Iceberg roundtrip on the streaming engine. You can scan an Iceberg table with scan_iceberg(), transform it lazily, and write the result back with sink_iceberg(). Useful for workflows like data redaction or compliance cleanup.

Polars@pola· Mar 31

Realtime query profiling of Polars In this post we use the query profiler in Polars Cloud to optimize the infrastructure configuration for a specific query. This results in a 54% faster and 64% cheaper query with only five runs. Read all about it here: pola.rs/posts/query-...

Polars@pola· Mar 26

We've released Polars Cloud client 0.6.0. Some of the highlights: • Improved UX for query profiling • Compute Scratchpad Alpha • Improved distributed query planning • Breaking: `LazyFrameRemote.execute` is now blocking by default

Polars@pola· Feb 26

pl.from_repr() constructs a DataFrame or Series directly from its printed string representation. This can be useful in unit tests: instead of rebuilding expected DataFrames through dictionaries with typecasting, the schema is encoded in the header and the values are right there in the table.

Polars@pola· Feb 13

str.len_bytes() vs str.len_chars() len_bytes: ~20x faster, counts UTF-8 bytes len_chars: counts actual Unicode characters - Use len_bytes for ASCII data (IDs, hashes) - Use len_chars for anything multilingual len_bytes is O(1) metadata lookup, len_chars is O(n) traversal.

Polars@pola· Jan 13

We just released Polars 1.37, here are the highlights: Improved Streaming Sinks: 1.14x-1.88x speedup, ~10% of the original memory. Streaming Compressed CSVs Faster SQL Ordering pl.PartitionBy min_by / max_by (see below) Series.sql() Free-Threading Support Python 3.9 Support Dropped musl Builds

Polars@pola· Dec 18

Did you know about pl.corr()? The problem with data aggregation is that it can hide what's really going on. Below you can find Simpson's Paradox Sometimes the devil really is in the details.

Polars@pola· Dec 11

"We adopted Polars to meet strict technical requirements, but the result went beyond simple optimization. The 30x performance improvement gave us the unexpected opportunity to do more." Read about how Rabobank deployed Polars in a critical enterprise production environment: lnkd.in/eZFPcxRw

Polars@pola· Dec 9

We've just released 1.36.0. Here are the highlights: Highlights: 🧩 Extension Types 🛟 Float16 Support ↪️ LazyFrame.pivot() 👀 DataFrame.show() 🗄️ SQL Parity: Added Window functions ⏱️ Parquet writer: 2.2x runtime improvement Find the full release notes here: github.com/pola-rs/pola...

Polars@pola· Nov 19

Polars recently shipped some performance upgrades and long-awaited features: 🏆 Decimal Type Now Stable 🏆 Aggregation over the List and Array Types ✨Other new features: Streaming ewm_mean() Expr.item() Expr.rolling_rank() pl.union() Read more: github.com/pola-rs/pola...

Polars@pola· Aug 13

Are you looking to get started with Polars over the summer? We've partnered with @datacamp.bsky.social to create an interactive course that covers the fundamentals so you can write your next query with Polars. The course is free till the end of August: www.datacamp.com/courses/intr...

Free DataCamp course announcement - Learn Polars, the high-performance data processing library that engineers love. Interactive Introduction to Polars course available at no cost. Partnership between DataCamp and Polars.

Polars@pola· May 28

We've partnered with @datacamp.bsky.social to create an interactive Polars course. Learn the fundamentals and get familiar with our API through hands-on exercises. The course is available for everyone and free until the end of August. Start the free course here: www.datacamp.com/courses/intr...

Course title slide with dark blue background showing 'FREE COURSE' in green text, 'Introduction to Polars' as the main white heading, and logos for DataCamp and Polars at the bottom.

Polars@pola· May 1

Polars has gotten 4x faster than Polars! 🚀 In the last months, the team has worked incredibly hard on the new-streaming engine and the results pay off. It is incredibly fast, and beats the Polars in-memory engine by a factor of 4 on a 96vCPU machine.

Polars@pola· Apr 30

Polars provides a number of xxx_horizontal operations. These expressions perform computations across columns. (Or along rows, depending on how you look at it.) If your horizontal operation isn’t implemented, you can use the general-purpose fold.

Diagram showing examples of applying horizontal operations to a dataframe with random numerical values. The example shows the expressions max_horizontal, sum_horizontal, mean_horizontal, and cum_sum_horizontal. The first three produce a numerical column and the expression cum_sum_horizontal produces a struct column with as many fields as there are input columns/series.

Polars@pola· Mar 19

Polars provides 3 functions you can use to generate temporal ranges: date_range, datetime_range, and time_range. These can be executed eagerly or lazily. You can also customize the interval between consecutive values and whether the start/end points are included.

Diagram showing how the functions date_range, datetime_range, and time_range, can be used to generate series of consecutive temporal values. The interval between consecutive values can be changed with the parameter interval and the two endpoints may or may not be included in the generated range, depending on the value the parameter closed is set to.

Polars@pola· Mar 11

It does:

Polars@pola· Feb 27

The expression over can be used to compute expressions within isolated groups. This means you can do computations per group without having to group first and then explode after. In this example, we rank swimmers based on their time, but within their race type.

Diagram showing how the use of the expression over impacts the result of an expression. Using rank on a dataframe will rank all swimmers by their time, but that means that the fastest swimmer of the second race type is actually ranked 3rd. By using over, the fastest swimmer of each race type gets the rank 1.

Polars@pola· Feb 20

The context filter lets you filter out rows from a dataframe based on some conditions. Within an aggregation, you can also use filter to filter values from aggregated groups. In this example we ignore unverified times when computing the current record.

Diagram showing how filter works inside aggregations. We use an aggregation to compute the current record for different swimming race types and we use a filter to ignore unverified times, while also using information about unverified times to compute another column.

Polars@pola· Feb 3

The expression `clip` is pretty straightforward: You provide a lower and an upper bound, and Polars makes sure all values fall within those bounds. If a value is too small/too large, it's replaced by the bound. Bounds can be literals, other columns, or arbitrary expressions.

Diagram showing how the expression clip works. The diagram shows `clip` used with integers and with `datetime` objects. The diagram also shows the usage of literals, other columns, and arbitrary expressions, as the bounds.

Polars@pola· Jan 7

Join our webinar with NVIDIA on January 28 for an in-depth session on how the GPU engine works, from collecting your query to parallel execution on the GPU. Sign up at info.nvidia.com/nvidia-polar... See you there?