Class

com.silectis.magpie

MagpieContext

Related Doc: package magpie

Permalink

final class MagpieContext extends LegacyVariableContext

Main entry point for Magpie functionality. The MagpieContext is used to execute commands, run SQL queries, get data frames for Magpie tables, and more.

Available as mc in scala script tasks and scala notebook blocks.

Example usage:

val flavors = Seq("apple", "banana", "strawberry")

flavors.foreach { f =>
  mc.sql(s"select * from store_sales where flavor = '$f'")
  mc.exec(s"save result as table ${f}_sales")
}
Linear Supertypes
LegacyVariableContext, MagpieStringUtils, SubstitutionUtils, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MagpieContext
  2. LegacyVariableContext
  3. MagpieStringUtils
  4. SubstitutionUtils
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def about: MagpieInfo

    Permalink

    Return information about the current Magpie session.

    Return information about the current Magpie session.

    returns

    Magpie info

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  9. def exec(command: String): Any

    Permalink

    Execute a single command and return the result.

    Execute a single command and return the result. Alias for execute.

    command

    command string to execute

    returns

    command result

    See also

    execute

  10. def execute(command: String): Any

    Permalink

    Execute a single command and return the result.

    Execute a single command and return the result.

    Commands return one of:

    mc.execute("save result as table people")
    command

    command string to execute

    returns

    command result

  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  13. def getTableDataFrame(table: String): DataFrame

    Permalink

    Get the data frame for a Magpie table, optionally qualified by schema.

    Get the data frame for a Magpie table, optionally qualified by schema.

    val df = mc.getTableDataFrame("people")
    df.show()
    table

    table name

    returns

    table data frame

  14. def getTokens(input: String): Seq[String]

    Permalink

    Returns the tokens that are potentially substitutable in the provided command.

    Returns the tokens that are potentially substitutable in the provided command.

    input

    a string containing 0 or more substitution tokens in the format "$#{type:name}". The legacy/shorthand format for variables is also supported ("$#varname")

    returns

    A de-duplicated list of tokens that are present 1 or more times in the provided command in the format "type:name". Variable tokens in the command that use the shorthand form are expanded to "variable:varname". Tokens are always lowercase

    Attributes
    protected
    Definition Classes
    SubstitutionUtils
  15. def hasTokens(input: String): Boolean

    Permalink

    Checks if the provided input contains any tokens

    Checks if the provided input contains any tokens

    input

    The string to test

    returns

    True if the input string contains 1 or more replacable tokens

    Attributes
    protected
    Definition Classes
    SubstitutionUtils
  16. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  17. def interpret(command: String): Unit

    Permalink

    Execute the provided command and render the result visually

    Execute the provided command and render the result visually

    For example, to render the first 100 rows of the table people:

    mc.interpret("show 100 from people")
    command

    command to execute

  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  19. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  22. def profile(df: DataFrame): Unit

    Permalink

    Profile a data frame and render the profile visually

    Profile a data frame and render the profile visually

    mc.profile(df)
    df

    data frame

  23. def replaceSubstitutesWithPlaceholders(input: String): String

    Permalink

    Replaces any substitute tokens in the given string with the token's name

    Replaces any substitute tokens in the given string with the token's name

    input

    a string containing 0 or more tokens

    returns

    a String with the tokens replaced by the name of the substitute reference by the token

    Attributes
    protected
    Definition Classes
    SubstitutionUtils
  24. def result: DataFrame

    Permalink

    Get the last result as a spark data frame

    Get the last result as a spark data frame

    mc.sql("select * from people")
    val people = mc.result.collect()
    returns

    result data frame

  25. val secrets: SecretContext

    Permalink

    SecretContext with methods for interacting with Magpie Secrets

  26. def sql(sql: String): DataFrame

    Permalink

    Execute a SQL command, returning the resulting data frame.

    Execute a SQL command, returning the resulting data frame.

    val df = mc.sql("select distinct name from people")
    sql

    SQL statement

    returns

    data frame result

  27. def substituteInCommand(command: String, substitutes: Seq[Substitute]): SubstitutionResult

    Permalink

    Inserts the provided substitutes into the provided command string.

    Inserts the provided substitutes into the provided command string. Script comments will be stripped from the returned string.

    Will throw a MagpieParsingException if no suitable substitute exists for a token

    command

    A string containing 0 or more substitute tokens, which may be in full or variable shorthand notation (See SubstitutionUtils.getTokens() for more details on token syntax)

    substitutes

    A sequence of unencrypted substitute objects

    returns

    A String with the substitute tokens replaced by their corresponding values

    Attributes
    protected
    Definition Classes
    SubstitutionUtils
  28. def substituteInText(text: String, substitutes: Seq[Substitute]): SubstitutionResult

    Permalink

    Inserts the provided substitutes into the provided text string

    Inserts the provided substitutes into the provided text string

    Will throw a MagpieParsingException if no suitable substitute exists for a token

    text

    A string containing 0 or more substitute tokens, which may be in full or variable shorthand notation (See SubstitutionUtils.getTokens() for more details on token syntax)

    substitutes

    A sequence of unencrypted substitute objects

    returns

    A String with the substitute tokens replaced by their corresponding values

    Attributes
    protected
    Definition Classes
    SubstitutionUtils
  29. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  30. def toCamelCase(s: String): String

    Permalink

    Converts CapitalCamelCase to camelCase

    Converts CapitalCamelCase to camelCase

    s

    to string to be converted

    returns

    a camelCase string

    Attributes
    protected
    Definition Classes
    MagpieStringUtils
  31. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  32. def toSubstitute(s: Secret): Substitute

    Permalink
    Definition Classes
    SubstitutionUtils
  33. def toSubstitute(v: Variable): Substitute

    Permalink
    Definition Classes
    SubstitutionUtils
  34. val variables: VariableContext

    Permalink

    VariableContext with methods for interacting with Magpie Variables

  35. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  37. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def clearVariable(name: String): Unit

    Permalink

    Unset a particular variable in the context.

    Unset a particular variable in the context.

    name

    variable name

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Moved to mc.variables.remove()

  2. def clearVariables(): Unit

    Permalink

    Clear all variables in the context.

    Clear all variables in the context.

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Moved to mc.variables.clear()

  3. def getVariable(name: String): Any

    Permalink

    Get the value of a variable.

    Get the value of a variable.

    name

    variable name

    returns

    variable value

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Moved to mc.variables.get()

  4. def hasVariable(name: String): Boolean

    Permalink

    Determine whether a variable is defined on the context.

    Determine whether a variable is defined on the context.

    name

    variable name

    returns

    whether the variable is defined

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Moved to mc.variables.exists()

  5. def printVariables(): String

    Permalink

    Get a listing of all variables defined on the context and their values.

    Get a listing of all variables defined on the context and their values.

    returns

    variable listing

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Use mc.variables.list() instead

  6. def setVariable(name: String, value: Any): Unit

    Permalink

    Set a variable to the given value in the context.

    Set a variable to the given value in the context.

    name

    variable name

    value

    variable value

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.21.0) Moved to mc.variables.set()

  7. def substituteVariables(input: String): String

    Permalink

    Substitute any variables present in the input string with current values stored in the context.

    Substitute any variables present in the input string with current values stored in the context.

    An exception is thrown if any variables are not able to be matched.

    Note: substitution is performed automatically on arguments passed to MagpieContext.execute, MagpieContext.exec, and MagpieContext.sql.

    input

    input string

    returns

    string with variables substituted for their values

    Definition Classes
    LegacyVariableContext
    Annotations
    @deprecated
    Deprecated

    (Since version v1.20.0) Variables are automatically substituted by the .exec, .execute, and .sql methods

Inherited from LegacyVariableContext

Inherited from MagpieStringUtils

Inherited from SubstitutionUtils

Inherited from AnyRef

Inherited from Any

Ungrouped