Back to directory
locationtech avatar

rasterframes

Geospatial Raster support for Spark DataFrames. Read more below about its uses, features, and usage.

Clone repository

git clone https://github.com/locationtech/rasterframes.git

256

Stars

48

Forks

11

Watchers

Apache-2.0

License

RasterFramesLogo®

Join the chat at https://gitter.im/locationtech/rasterframes

RasterFrames® brings together Earth-observation (EO) data access, cloud computing, and DataFrame-based data science. The recent explosion of EO data from public and private satellite operators presents both a huge opportunity as well as a challenge to the data analysis community. It is Big Data in the truest sense, and its footprint is rapidly getting bigger.

RasterFrames provides a DataFrame-centric view over arbitrary raster data, enabling spatiotemporal queries, map algebra raster operations, and compatibility with the ecosystem of Spark ML algorithms. By using DataFrames as the core cognitive and compute data model, it is able to deliver these features in a form that is both accessible to general analysts and scalable along with the rapidly growing data footprint.

rasterframes pipeline nologo

Please see the Getting Started section of the Users' Manual to start using RasterFrames.

User Resources

Contributing

Community contributions are always welcome. To get started, please review our contribution guidelines, code of conduct, and reach out to us on gitter so the community can help you get started!

RasterFrames is part of the LocationTech Stack.

rasterframes locationtech stack

It is written in Scala, but with Python bindings. If you wish to contribute to the development of RasterFrames, or you wish to build it from scratch, you will need sbt. Then clone the repository from GitHub.

git clone https://github.com/locationtech/rasterframes.git
cd rasterframes

To publish to your local repository:

sbt publishLocal

You can run tests with

sbt test

and integration tests

sbt it:test

The documentation may be built with

sbt makeSite

Additional, Python sepcific build instruction may be found at pyrasterframes/src/main/python/README.md

RasterFrames is released under the commercial-friendly Apache 2.0 License, copyright Astraea, Inc. 2017-2021.

Commercial Support

As the sponsors and developers of RasterFrames, Astraea, Inc. is uniquely positioned to expand its capabilities. If you need additional functionality or just some architectural guidance to get your project off to the right start, we can provide a full range of consulting and development services around RasterFrames. We can be reached at [email protected].

Releases

What's Changed

Update to frameless 0.11 by @pomadchin in #574 Update to Spark 3.3 by @echeipesh in #587 #598 Improve Python build process and streamline CI by @thomas-maschler in #600 Match Scala dep...

Mar 14, 2022

Download .zip

What's Changed

Fix UDTs registration ordering by @pomadchin in #573 Ability to write RasterFrame tile columns directly as raster files. by @metasim in #571 Bump frameless version up by @pomadchin in...

Oct 15, 2021

Download .zip

Major Updates

Upgraded to Scala 2.12 , Spark 3.1.2, and GeoTrellis 3.6.0 (a subtantial accomplishment!) Added buffered tile support Added focal operations: rf_focal_mean, rf_focal_median,rf_focal_mod...

Upgraded to Spark 2.4.7 Added pyspark.sql.DataFrame.display(num_rows:int, truncate:bool) extension method when rf_ipython is imported. Added users' manual section on IPython display enhancements. Adde...

Apr 23, 2020

Download .zip

Upgraded to GeoTrellis 3.3.0. This includes a number of breaking changes enumerated as a part of the PR's change log. These include:

Add Int type parameter to Grid Add Int type parameter to CellGrid...