org.apache.spark

Class TestUtils

Object

org.apache.spark.TestUtils

public class TestUtils
extends Object

Utilities for tests. Included in main codebase since it's used by multiple projects.

TODO: See if we can move this to the test codebase by specifying test dependencies between projects.

Constructor Summary

Constructors

Constructor and Description
TestUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static <T> void	assertNotSpilled(SparkContext sc, String identifier, scala.Function0<T> body) Run some code involving jobs submitted to the given context and assert that the jobs did not spill.
static <T> void	assertSpilled(SparkContext sc, String identifier, scala.Function0<T> body) Run some code involving jobs submitted to the given context and assert that the jobs spilled.
static java.io.File	createCompiledClass(String className, java.io.File destDir, String toStringValue, String baseClass, scala.collection.Seq<java.net.URL> classpathUrls) Creates a compiled class with the given name.
static java.io.File	createCompiledClass(String className, java.io.File destDir, org.apache.spark.TestUtils.JavaSourceFromString sourceFile, scala.collection.Seq<java.net.URL> classpathUrls) Creates a compiled class with the source file.
static java.net.URL	createJar(scala.collection.Seq<java.io.File> files, java.io.File jarFile, scala.Option<String> directoryPrefix) Create a jar file that contains this set of files.
static java.net.URL	createJarWithClasses(scala.collection.Seq<String> classNames, String toStringValue, scala.collection.Seq<scala.Tuple2<String,String>> classNamesWithBase, scala.collection.Seq<java.net.URL> classpathUrls) Create a jar that defines classes with the given names.
static java.net.URL	createJarWithFiles(scala.collection.immutable.Map<String,String> files, java.io.File dir) Create a jar file containing multiple files.
static int	httpResponseCode(java.net.URL url, String method, scala.collection.Seq<scala.Tuple2<String,String>> headers) Returns the response code from an HTTP(S) URL.
static scala.Tuple2<Object,scala.Option<String>>	httpResponseCodeAndURL(java.net.URL url, String method, scala.collection.Seq<scala.Tuple2<String,String>> headers) Returns the response code and url (if redirected) from an HTTP(S) URL.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestUtils

public TestUtils()

Method Detail

createJarWithClasses

public static java.net.URL createJarWithClasses(scala.collection.Seq<String> classNames,
                                String toStringValue,
                                scala.collection.Seq<scala.Tuple2<String,String>> classNamesWithBase,
                                scala.collection.Seq<java.net.URL> classpathUrls)

Create a jar that defines classes with the given names.

Note: if this is used during class loader tests, class names should be unique in order to avoid interference between tests.

Parameters:

classNames - (undocumented)

toStringValue - (undocumented)

classNamesWithBase - (undocumented)

classpathUrls - (undocumented)

Returns:

(undocumented)

createJarWithFiles

public static java.net.URL createJarWithFiles(scala.collection.immutable.Map<String,String> files,
                              java.io.File dir)

Create a jar file containing multiple files. The files map contains a mapping of file names in the jar file to their contents.

Parameters:

files - (undocumented)

dir - (undocumented)

Returns:

(undocumented)

createJar

public static java.net.URL createJar(scala.collection.Seq<java.io.File> files,
                     java.io.File jarFile,
                     scala.Option<String> directoryPrefix)

Create a jar file that contains this set of files. All files will be located in the specified directory or at the root of the jar.

Parameters:

files - (undocumented)

jarFile - (undocumented)

directoryPrefix - (undocumented)

Returns:

(undocumented)

createCompiledClass

public static java.io.File createCompiledClass(String className,
                               java.io.File destDir,
                               org.apache.spark.TestUtils.JavaSourceFromString sourceFile,
                               scala.collection.Seq<java.net.URL> classpathUrls)

Creates a compiled class with the source file. Class file will be placed in destDir.

createCompiledClass

public static java.io.File createCompiledClass(String className,
                               java.io.File destDir,
                               String toStringValue,
                               String baseClass,
                               scala.collection.Seq<java.net.URL> classpathUrls)

Creates a compiled class with the given name. Class file will be placed in destDir.

assertSpilled

public static <T> void assertSpilled(SparkContext sc,
                     String identifier,
                     scala.Function0<T> body)

Run some code involving jobs submitted to the given context and assert that the jobs spilled.

Parameters:

sc - (undocumented)

identifier - (undocumented)

body - (undocumented)

assertNotSpilled

public static <T> void assertNotSpilled(SparkContext sc,
                        String identifier,
                        scala.Function0<T> body)

Run some code involving jobs submitted to the given context and assert that the jobs did not spill.

Parameters:

sc - (undocumented)

identifier - (undocumented)

body - (undocumented)

httpResponseCodeAndURL

public static scala.Tuple2<Object,scala.Option<String>> httpResponseCodeAndURL(java.net.URL url,
                                                               String method,
                                                               scala.collection.Seq<scala.Tuple2<String,String>> headers)

Returns the response code and url (if redirected) from an HTTP(S) URL.

Parameters:

url - (undocumented)

method - (undocumented)

headers - (undocumented)

Returns:

(undocumented)

httpResponseCode

public static int httpResponseCode(java.net.URL url,
                   String method,
                   scala.collection.Seq<scala.Tuple2<String,String>> headers)

Returns the response code from an HTTP(S) URL.

Parameters:

url - (undocumented)

method - (undocumented)

headers - (undocumented)

Returns:

(undocumented)