de.hybris.platform.impex.jalo

Class Importer

java.lang.Object

de.hybris.platform.impex.jalo.Importer

All Implemented Interfaces:

ImpExLogFilter.LocationProvider

Direct Known Subclasses:

MultiThreadedImporter

public class Importer
extends Object
implements ImpExLogFilter.LocationProvider

Central class for processing an import. An import process is always based on a CSV-stream with special ImpEx syntax (see ImpEx Documentation for how to define an ImpEx CSV-file).
The process is separated in 3 steps:

1. Instantiation:
   While instantiating the Importer class, this CSV-stream will be given using an CSVReader or an ImpExImportReader. If you only want to specify the input stream use an CSVReader, the Importer will instantiate an corresponding ImpExImportReader. The usage of an ImpExImportReader is only preferred, if special settings while instantiation are needed (settings after instantiation can be done using the getReader() method of the Importer instance).
   
   Example:
   CSVReader reader = new CSVReader( "input.csv", "utf-8" );
Importer importer = new Importer( reader );
or
   Importer importer = new Importer( new ImpExImportReader( reader, new MyImportProcessor() ) );

2. Configuration:
   You have several possibilities for configuring the import process.
   First you can configure the used ImpExImportReader using the getReader method. Here you can configure several things about the reading of the input (skipValueLines, enableCodeExecution, ..) or item processing ( setRelaxedMode, .. ).
   Second, you can set an DumpHandler for specifying the dump file handling.
   Third, you can set an ErrorHandler to specify the process in case of an error.
   Fourth, you can set the maximal amount of passes for resolving dumped value lines.
3. Import:
   The import can be performed using the importNext which processes the input stream until an item was processed (insert, update or remove). It returns the processed item.
   An other possibility is the usage of the importAll method which calls the importNext method until finishing of the input stream. While and after the import process you have several possibilities to get information about the state (current pass, processed items .. ).
   
   Example:
   Item item = null;
do
{
item = importer.importNext();
System.out.println( "Processed items: " + getProcessedItemsCountOverall() );
}
while( item != null );
or
   importer.importAll();
System.out.println( "Processed items: " + getProcessedItemsCountOverall() );


Field Summary

Fields

Modifier and Type	Field and Description
static long	LOG_INTERVAL Duration between two logs.

Constructor Summary

Constructors

Constructor and Description
Importer(CSVReader source) Instantiates a Importer using given source as input for import.
Importer(ImpExImportReader importReader) Instantiates a Importer using given importReader for reading input and item processing.

Method Summary

Modifier and Type	Method and Description
void	abortImport() Aborts the import phase by closing all readers and writers, cleaning dump files and switching to finish state.
protected void	activateLogFilter()
protected void	assureStates(de.hybris.platform.impex.jalo.Importer.STATE... states) Assures that the instance is in one of given states.
void	close() Closes the used reader instance.
protected ImpExImportReader	createImportReader(CSVReader csvReader) Creates an ImpExImportReader instance for processing given input.
protected ImpExImportReader	createImportReaderForNextPass() Creates an ImpExImportReader instance for next pass using dump handler for getting input stream and dump stream.
protected void	deactivateLogFilter()
protected Item	doImport() Performs the readLine method on used ImpExImportReader instance and returns read item.
protected void	finishImport() Finishes the import phase by closing all readers and writers, cleaning dump files and switching state.
HeaderDescriptor	getCurrentHeader() Gets the current header descriptor processed by input reader.
int	getCurrentLineNumber() Gets the current line number in input stream.
String	getCurrentLocation() Gets the current location within input stream as text representation.
int	getCurrentPass() Gets the number of the current pass, the import process performs.
int	getDumpedLineCountOverall() Returns the amount of lines written to the dump data file in all passes.
int	getDumpedLineCountPerHeader() Returns the amount of lines of current header written to the dump data file.
int	getDumpedLineCountPerPass() Returns the amount of lines written to the dump data file in current pass intended to be processed within a second import run or more.
DumpHandler	getDumpHandler() Gets the used DumpHandler instance for handling dump medias.
ErrorHandler	getErrorHandler() Gets the used ErrorHandler instance for handling exceptions while import.
int	getLastImportedItemLineNumber() Gets the line number of last imported line at current input.
ImpExLogFilter	getLogFilter()
int	getMaxPass() Gets the configured amount of maximal passes the import will perform for unresolving value lines.
int	getProcessedItemsCountOverall() Returns the overall processed item count.
int	getProcessedItemsCountPerHeader() Returns the overall processed item count of current header.
int	getProcessedItemsCountPerPass() Returns the overall processed item count of current pass.
ImpExImportReader	getReader() Gets the used ImpExImportReader instance.
int	getResolvedItemsCountOverall() Returns the overall item count processed and not dumped (means all dependencies are resolved).
int	getResolvedItemsCountPerPass() Returns the item count processed and not dumped in current pass (means all dependencies are resolved).
int	getValueLineCountPerPass() Returns the overall value line count of current pass.
boolean	hadError()
boolean	hasUnresolvedLines() Checks if there are unresolved lines written to dump.
void	importAll() The import process will be performed from input stream given in constructor.
Item	importNext() Tries to import the next item from input stream.
protected Item	importNextInternal()
protected void	init() Initializes the instance and switches to running state.
boolean	isAborted() Checks if instance has finished reading from input (has finished import process).
boolean	isFinished() Checks if instance has finished reading from input (has finished import process).
boolean	isRunning() Checks if instance is in import phase currently.
protected void	logProcess() Logs current state to the log4j system.
protected boolean	prepareNextPass() After checking the need for a next pass, it prepares the next import cycle by calling prepareNextPass of used DumpHandler and setting old dump file as input.
void	setDumpHandler(DumpHandler handler) Sets the DumpHandler to use while import process.
void	setErrorHandler(ErrorHandler handler) Sets the ErrorHandler to use while import.
void	setLogFilter(ImpExLogFilter filter) Sets the logging filter to use while import.
void	setMaxPass(int maxPass) Sets the maximal amount of passes the import process will perform.
protected void	switchToState(de.hybris.platform.impex.jalo.Importer.STATE newState) Sets the curSteate flag to given state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG_INTERVAL

public static final long LOG_INTERVAL

Duration between two logs.

See Also:

Constant Field Values

Constructor Detail

Importer

public Importer(CSVReader source)

Instantiates a Importer using given source as input for import.

Parameters:

source - encapsulated CSV-stream

Importer

public Importer(ImpExImportReader importReader)

Instantiates a Importer using given importReader for reading input and item processing.

Parameters:

importReader - reader used for input reading and item processing

Method Detail

init

protected void init()
             throws ImpExException

Initializes the instance and switches to running state.

Throws:

ImpExException - error while dump file creation

switchToState

protected void switchToState(de.hybris.platform.impex.jalo.Importer.STATE newState)

Sets the curSteate flag to given state.

Parameters:

newState - new state to which the current state will be set

logProcess

protected void logProcess()

Logs current state to the log4j system. Will log only if log level and log interval fits.

doImport

protected Item doImport()
                 throws ImpExException

Performs the readLine method on used ImpExImportReader instance and returns read item. If set ErrorHandler evaluated an error to ignore, the next item will be read.

Returns:

read item or null

Throws:

ImpExException - error while reading from input or item processing

prepareNextPass

protected boolean prepareNextPass()
                           throws ImpExException

After checking the need for a next pass, it prepares the next import cycle by calling prepareNextPass of used DumpHandler and setting old dump file as input.

Returns:

true if import process is ready to continue with next pass (input is configured and ready for reading), false if a next pass is not needed

Throws:

ImpExException - error while preparing dump files

finishImport

protected void finishImport()

Finishes the import phase by closing all readers and writers, cleaning dump files and switching state.

abortImport

public void abortImport()

Aborts the import phase by closing all readers and writers, cleaning dump files and switching to finish state.

assureStates

protected void assureStates(de.hybris.platform.impex.jalo.Importer.STATE... states)

Assures that the instance is in one of given states. Throws IllegalStateException if instance is not in one of given states currently.

Parameters:

states - states to check for

createImportReader

protected ImpExImportReader createImportReader(CSVReader csvReader)

Creates an ImpExImportReader instance for processing given input.

Parameters:

csvReader - reader covering input stream

Returns:

new instance

createImportReaderForNextPass

protected ImpExImportReader createImportReaderForNextPass()

Creates an ImpExImportReader instance for next pass using dump handler for getting input stream and dump stream.

Returns:

new reader instance for processing next pass

isRunning

public boolean isRunning()

Checks if instance is in import phase currently.

Returns:

true if instance is in import phase, else false

isFinished

public boolean isFinished()

Checks if instance has finished reading from input (has finished import process).

Returns:

true if instance has finished reading from input, else false

isAborted

public boolean isAborted()

Checks if instance has finished reading from input (has finished import process).

Returns:

true if instance has finished reading from input, else false

getMaxPass

public int getMaxPass()

Gets the configured amount of maximal passes the import will perform for unresolving value lines.

Returns:

amount of maximal passes

getCurrentPass

public int getCurrentPass()

Gets the number of the current pass, the import process performs.

Returns:

current pass number

getReader

public ImpExImportReader getReader()

Gets the used ImpExImportReader instance.

Returns:

used reader instance

getDumpHandler

public DumpHandler getDumpHandler()

Gets the used DumpHandler instance for handling dump medias.

Returns:

used dump handler instance

getErrorHandler

public ErrorHandler getErrorHandler()

Gets the used ErrorHandler instance for handling exceptions while import.

Returns:

used error handler

hasUnresolvedLines

public boolean hasUnresolvedLines()

Checks if there are unresolved lines written to dump.

Returns:

true if there are unresolved lines, otherwise false

getValueLineCountPerPass

public int getValueLineCountPerPass()

Returns the overall value line count of current pass. This count includes all value lines which are read, including skipped, unresolved or otherwise unprocessable lines!

Returns:

the overall value line count of current pass

getProcessedItemsCountPerHeader

public int getProcessedItemsCountPerHeader()

Returns the overall processed item count of current header.

Returns:

overall processed item count of current header

getProcessedItemsCountPerPass

public int getProcessedItemsCountPerPass()

Returns the overall processed item count of current pass. Equals getValueCountPerPass without skipped and discarded value lines.

Returns:

the overall processed item count of current pass

getProcessedItemsCountOverall

public int getProcessedItemsCountOverall()

Returns the overall processed item count.

Returns:

the overall processed item count

getDumpedLineCountPerHeader

public int getDumpedLineCountPerHeader()

Returns the amount of lines of current header written to the dump data file.

Returns:

amount of lines of current header written to the dump data file

getResolvedItemsCountPerPass

public int getResolvedItemsCountPerPass()

Returns the item count processed and not dumped in current pass (means all dependencies are resolved).

Returns:

overall item count processed and not dumped in current pass

getResolvedItemsCountOverall

public int getResolvedItemsCountOverall()

Returns the overall item count processed and not dumped (means all dependencies are resolved).

Returns:

overall item count processed and not dumped in current pass

getDumpedLineCountPerPass

public int getDumpedLineCountPerPass()

Returns the amount of lines written to the dump data file in current pass intended to be processed within a second import run or more.
If this method returns 0 all value lines of current pass should have been processed properly and no second pass is necessary.

Returns:

amount of lines written to the dump data file in current pass

getDumpedLineCountOverall

public int getDumpedLineCountOverall()

Returns the amount of lines written to the dump data file in all passes.
ATTENTION: A value greater 0 means that there was a pass with dumped lines, it does not say anything about the current pass exclusively.

Returns:

amount of lines written to the dump data file in current pass

getCurrentLineNumber

public int getCurrentLineNumber()

Gets the current line number in input stream.

Returns:

current line number of input

See Also:

CSVReader.getCurrentLineNumber()

getLastImportedItemLineNumber

public int getLastImportedItemLineNumber()

Gets the line number of last imported line at current input.

Returns:

current line number of last imported line of input

getCurrentLocation

public String getCurrentLocation()

Gets the current location within input stream as text representation.

Specified by:

getCurrentLocation in interface ImpExLogFilter.LocationProvider

Returns:

current location text

See Also:

ImpExReader.getCurrentLocation()

getCurrentHeader

public HeaderDescriptor getCurrentHeader()

Gets the current header descriptor processed by input reader.

Returns:

current header

See Also:

ImpExReader.getCurrentHeader()

importAll

public void importAll()
               throws ImpExException

The import process will be performed from input stream given in constructor. Before calling this method you can configure the DumpHandler, the ErrorHandler and the maximal amount of passes. The call will automatically perform the whole import process including the dumped line processing (if maximal passes are a positive number). You can call the importNext method too for importing item based. This method is only a convenience kind which calls the importNext method until it returns null.

Throws:

ImpExException - an error is occurred while import processing (reasoned by input stream errors, dump file handling or item processing)

importNext

public final Item importNext()
                      throws ImpExException

Tries to import the next item from input stream. If called for one time, you can not call set.. methods anymore. If the input stream ends and there are unresolved lines, the call will automatically start the next pass by using the dump file as new input stream.

Returns:

the last imported item

Throws:

ImpExException - ImpExException an error is occurred while import processing (reasoned by input stream errors, dump file handling or item processing)

importNextInternal

protected Item importNextInternal()
                           throws ImpExException

Throws:

ImpExException

activateLogFilter

protected void activateLogFilter()

deactivateLogFilter

protected void deactivateLogFilter()

close

public void close()

Closes the used reader instance. Is not needed to call if the import process has finished ( isFinished() returns true).

setMaxPass

public void setMaxPass(int maxPass)

Sets the maximal amount of passes the import process will perform. Use -1 for infinit loops (will break if no changes occur anymore).

Parameters:

maxPass - new amount of maximal passes

setDumpHandler

public void setDumpHandler(DumpHandler handler)

Sets the DumpHandler to use while import process.

Parameters:

handler - new dump handler instance

setErrorHandler

public void setErrorHandler(ErrorHandler handler)

Sets the ErrorHandler to use while import.

Parameters:

handler - new error handler instance

setLogFilter

public void setLogFilter(ImpExLogFilter filter)

Sets the logging filter to use while import. The default one extends each message with the current reader location except for info logs.

Parameters:

filter - new error handler instance

getLogFilter

public ImpExLogFilter getLogFilter()

hadError

public boolean hadError()