Package org.apache.sedona.core.spatialPartitioning

Class BroadcastedSpatialPartitioner

java.lang.Object
org.apache.spark.Partitioner
org.apache.sedona.core.spatialPartitioning.SpatialPartitioner
org.apache.sedona.core.spatialPartitioning.BroadcastedSpatialPartitioner
All Implemented Interfaces:
Serializable
public class BroadcastedSpatialPartitioner extends SpatialPartitioner
The SpatialPartitioner may contain a large number of grids, which may make the serialized tasks to be larger than 1MB and trigger a warning: "WARN DAGScheduler: Broadcasting large task binary with size XXXX KB". This class is a wrapper around a SpatialPartitioner that is broadcasted to reduce the size of serialized tasks.
See Also:

  • Constructor Details

    • BroadcastedSpatialPartitioner

      public BroadcastedSpatialPartitioner(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)

  • Method Details

    • placeObject

      public <T extends org.locationtech.jts.geom.Geometry> Iterator<scala.Tuple2<Integer,T>> placeObject(T spatialObject) throws Exception
      Description copied from class: SpatialPartitioner
      Given a geometry, returns a list of partitions it overlaps.

      For points, returns exactly one partition as long as grid type is non-overlapping. For other geometry types or for overlapping grid types, may return multiple partitions.

      Specified by:
      placeObject in class SpatialPartitioner
      Throws:
      Exception

    • getDedupParams

      @Nullable public DedupParams getDedupParams()
      Specified by:
      getDedupParams in class SpatialPartitioner

    • getGrids

      public List<org.locationtech.jts.geom.Envelope> getGrids()
      Overrides:
      getGrids in class SpatialPartitioner

    • numPartitions

      public int numPartitions()
      Specified by:
      numPartitions in class org.apache.spark.Partitioner