Class

com.silectis.magpie.runtime

MagpieRows

Related Doc: package runtime

Permalink

case class MagpieRows(schema: StructType, rows: Array[Row] = Array(), totalCount: Option[Long] = None) extends MagpieResultSet with SparkSchemaUtils with Product with Serializable

A collection of rows returned from a magpie command, optionally including the total count of the source data set.

schema

the schema of the source data set

rows

the spark rows returned by the command, with columns converted to strings for display

totalCount

the total size of the source data set for the command

Linear Supertypes
Serializable, Serializable, Product, Equals, SparkSchemaUtils, FieldUtils, TableUtils, LazyLogging, MagpieResultSet, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MagpieRows
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. SparkSchemaUtils
  7. FieldUtils
  8. TableUtils
  9. LazyLogging
  10. MagpieResultSet
  11. AnyRef
  12. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new MagpieRows(schema: StructType, rows: Array[Row] = Array(), totalCount: Option[Long] = None)

    Permalink

    schema

    the schema of the source data set

    rows

    the spark rows returned by the command, with columns converted to strings for display

    totalCount

    the total size of the source data set for the command

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. def count: Int

    Permalink

    The number of rows returned by the command

    The number of rows returned by the command

    returns

    number of rows

    Definition Classes
    MagpieRows → MagpieResultSet

  7. def createReadSchema(structure: DataStructure): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  8. def createStructType(fields: Seq[Field]): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  9. def createWriteSchema(table: Table): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  10. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  13. def getFieldType(t: DataType): FieldType

    Permalink
    Definition Classes
    FieldUtils

  14. def getSparkType(fieldType: FieldType): DataType

    Permalink
    Attributes
    protected
    Definition Classes
    FieldUtils

  15. def getWritableFields(table: Table): Seq[Field]

    Permalink
    Attributes
    protected
    Definition Classes
    TableUtils

  16. def header: Seq[String]

    Permalink
    Attributes
    protected
    Definition Classes
    MagpieRows → MagpieResultSet

  17. def isDirectMap(mapping: Map[String, String]): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  19. lazy val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    LazyLogging

  20. def mapDfColumns(sourceDf: DataFrame, structure: DataStructure, mapping: Map[String, String]): DataFrame

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  21. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  22. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  23. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  24. def readFieldMetadata(schema: StructType): Seq[Field]

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  25. def reverseMapDfColumns(df: DataFrame, mapping: Map[String, String]): DataFrame

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  26. val rows: Array[Row]

    Permalink

    the spark rows returned by the command, with columns converted to strings for display

  27. val schema: StructType

    Permalink

    the schema of the source data set

  28. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  29. val totalCount: Option[Long]

    Permalink

    the total size of the source data set for the command

    the total size of the source data set for the command

    Definition Classes
    MagpieRows → MagpieResultSet

  30. def validateSchema(actual: StructType, expected: StructType): Seq[String]

    Permalink
    Attributes
    protected
    Definition Classes
    SparkSchemaUtils

  31. def validateSchema(actual: Seq[Field], expected: Seq[Field]): Seq[String]

    Permalink
    Attributes
    protected
    Definition Classes
    TableUtils

  32. def validateTableDelete(mapping: Option[PersistenceMapping]): Future[Unit]

    Permalink
    Attributes
    protected
    Definition Classes
    TableUtils

  33. def values: Seq[Seq[String]]

    Permalink
    Attributes
    protected
    Definition Classes
    MagpieRows → MagpieResultSet

  34. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  35. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  36. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from SparkSchemaUtils

Inherited from FieldUtils

Inherited from TableUtils

Inherited from LazyLogging

Inherited from MagpieResultSet

Inherited from AnyRef

Inherited from Any

Ungrouped