Package de.hybris.platform.impex.jalo

Class Importer

  • All Implemented Interfaces:
    ImpExLogFilter.LocationProvider
    Direct Known Subclasses:
    MultiThreadedImporter
    public class Importer
extends java.lang.Object
implements ImpExLogFilter.LocationProvider
    Central class for processing an import. An import process is always based on a CSV-stream with special ImpEx syntax (see ImpEx Documentation for how to define an ImpEx CSV-file).
    The process is separated in 3 steps:
    1. Instantiation:
      While instantiating the Importer class, this CSV-stream will be given using an CSVReader or an ImpExImportReader. If you only want to specify the input stream use an CSVReader, the Importer will instantiate an corresponding ImpExImportReader. The usage of an ImpExImportReader is only preferred, if special settings while instantiation are needed (settings after instantiation can be done using the getReader() method of the Importer instance).

      Example:
      CSVReader reader = new CSVReader( "input.csv", "utf-8" );
      Importer importer = new Importer( reader );
      or
      Importer importer = new Importer( new ImpExImportReader( reader, new MyImportProcessor() ) );
    2. Configuration:
      You have several possibilities for configuring the import process.
      First you can configure the used ImpExImportReader using the getReader method. Here you can configure several things about the reading of the input (skipValueLines, enableCodeExecution, ..) or item processing ( setRelaxedMode, .. ).
      Second, you can set an DumpHandler for specifying the dump file handling.
      Third, you can set an ErrorHandler to specify the process in case of an error.
      Fourth, you can set the maximal amount of passes for resolving dumped value lines.
    3. Import:
      The import can be performed using the importNext which processes the input stream until an item was processed (insert, update or remove). It returns the processed item.
      An other possibility is the usage of the importAll method which calls the importNext method until finishing of the input stream. While and after the import process you have several possibilities to get information about the state (current pass, processed items .. ).

      Example:
      Item item = null;
      do
      {
      item = importer.importNext();
      System.out.println( "Processed items: " + getProcessedItemsCountOverall() );
      }
      while( item != null );
      or
      importer.importAll();
      System.out.println( "Processed items: " + getProcessedItemsCountOverall() );

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static long LOG_INTERVAL
      Duration between two logs.

    • Constructor Summary

      Constructors 
      Constructor Description
      Importer​(ImpExImportReader importReader)
      Instantiates a Importer using given importReader for reading input and item processing.
      Importer​(CSVReader source)
      Instantiates a Importer using given source as input for import.

    • Field Detail

      • LOG_INTERVAL

        public static final long LOG_INTERVAL
        Duration between two logs.
        See Also:
        Constant Field Values

    • Constructor Detail

      • Importer

        public Importer​(CSVReader source)
        Instantiates a Importer using given source as input for import.
        Parameters:
        source - encapsulated CSV-stream

      • Importer

        public Importer​(ImpExImportReader importReader)
        Instantiates a Importer using given importReader for reading input and item processing.
        Parameters:
        importReader - reader used for input reading and item processing

    • Method Detail

      • init

        protected void init()
             throws ImpExException
        Initializes the instance and switches to running state.
        Throws:
        ImpExException - error while dump file creation

      • switchToState

        protected void switchToState​(de.hybris.platform.impex.jalo.Importer.STATE newState)
        Sets the curSteate flag to given state.
        Parameters:
        newState - new state to which the current state will be set

      • logProcess

        protected void logProcess()
        Logs current state to the log4j system. Will log only if log level and log interval fits.

      • doImport

        protected Item doImport()
                 throws ImpExException
        Performs the readLine method on used ImpExImportReader instance and returns read item. If set ErrorHandler evaluated an error to ignore, the next item will be read.
        Returns:
        read item or null
        Throws:
        ImpExException - error while reading from input or item processing

      • prepareNextPass

        protected boolean prepareNextPass()
                           throws ImpExException
        After checking the need for a next pass, it prepares the next import cycle by calling prepareNextPass of used DumpHandler and setting old dump file as input.
        Returns:
        true if import process is ready to continue with next pass (input is configured and ready for reading), false if a next pass is not needed
        Throws:
        ImpExException - error while preparing dump files

      • finishImport

        protected void finishImport()
        Finishes the import phase by closing all readers and writers, cleaning dump files and switching state.

      • abortImport

        public void abortImport()
        Aborts the import phase by closing all readers and writers, cleaning dump files and switching to finish state.

      • assureStates

        protected void assureStates​(de.hybris.platform.impex.jalo.Importer.STATE... states)
        Assures that the instance is in one of given states. Throws IllegalStateException if instance is not in one of given states currently.
        Parameters:
        states - states to check for

      • createImportReader

        protected ImpExImportReader createImportReader​(CSVReader csvReader)
        Creates an ImpExImportReader instance for processing given input.
        Parameters:
        csvReader - reader covering input stream
        Returns:
        new instance

      • createImportReaderForNextPass

        protected ImpExImportReader createImportReaderForNextPass()
        Creates an ImpExImportReader instance for next pass using dump handler for getting input stream and dump stream.
        Returns:
        new reader instance for processing next pass

      • isRunning

        public boolean isRunning()
        Checks if instance is in import phase currently.
        Returns:
        true if instance is in import phase, else false

      • isFinished

        public boolean isFinished()
        Checks if instance has finished reading from input (has finished import process).
        Returns:
        true if instance has finished reading from input, else false

      • isAborted

        public boolean isAborted()
        Checks if instance has finished reading from input (has finished import process).
        Returns:
        true if instance has finished reading from input, else false

      • getMaxPass

        public int getMaxPass()
        Gets the configured amount of maximal passes the import will perform for unresolving value lines.
        Returns:
        amount of maximal passes

      • getCurrentPass

        public int getCurrentPass()
        Gets the number of the current pass, the import process performs.
        Returns:
        current pass number

      • getReader

        public ImpExImportReader getReader()
        Gets the used ImpExImportReader instance.
        Returns:
        used reader instance

      • getDumpHandler

        public DumpHandler getDumpHandler()
        Gets the used DumpHandler instance for handling dump medias.
        Returns:
        used dump handler instance

      • getErrorHandler

        public ErrorHandler getErrorHandler()
        Gets the used ErrorHandler instance for handling exceptions while import.
        Returns:
        used error handler

      • hasUnresolvedLines

        public boolean hasUnresolvedLines()
        Checks if there are unresolved lines written to dump.
        Returns:
        true if there are unresolved lines, otherwise false

      • getValueLineCountPerPass

        public int getValueLineCountPerPass()
        Returns the overall value line count of current pass. This count includes all value lines which are read, including skipped, unresolved or otherwise unprocessable lines!
        Returns:
        the overall value line count of current pass

      • getProcessedItemsCountPerHeader

        public int getProcessedItemsCountPerHeader()
        Returns the overall processed item count of current header.
        Returns:
        overall processed item count of current header

      • getProcessedItemsCountPerPass

        public int getProcessedItemsCountPerPass()
        Returns the overall processed item count of current pass. Equals getValueCountPerPass without skipped and discarded value lines.
        Returns:
        the overall processed item count of current pass

      • getProcessedItemsCountOverall

        public int getProcessedItemsCountOverall()
        Returns the overall processed item count.
        Returns:
        the overall processed item count

      • getDumpedLineCountPerHeader

        public int getDumpedLineCountPerHeader()
        Returns the amount of lines of current header written to the dump data file.
        Returns:
        amount of lines of current header written to the dump data file

      • getResolvedItemsCountPerPass

        public int getResolvedItemsCountPerPass()
        Returns the item count processed and not dumped in current pass (means all dependencies are resolved).
        Returns:
        overall item count processed and not dumped in current pass

      • getResolvedItemsCountOverall

        public int getResolvedItemsCountOverall()
        Returns the overall item count processed and not dumped (means all dependencies are resolved).
        Returns:
        overall item count processed and not dumped in current pass

      • getDumpedLineCountPerPass

        public int getDumpedLineCountPerPass()
        Returns the amount of lines written to the dump data file in current pass intended to be processed within a second import run or more.
        If this method returns 0 all value lines of current pass should have been processed properly and no second pass is necessary.
        Returns:
        amount of lines written to the dump data file in current pass

      • getDumpedLineCountOverall

        public int getDumpedLineCountOverall()
        Returns the amount of lines written to the dump data file in all passes.
        ATTENTION: A value greater 0 means that there was a pass with dumped lines, it does not say anything about the current pass exclusively.
        Returns:
        amount of lines written to the dump data file in current pass

      • getCurrentLineNumber

        public int getCurrentLineNumber()
        Gets the current line number in input stream.
        Returns:
        current line number of input
        See Also:
        CSVReader.getCurrentLineNumber()

      • getLastImportedItemLineNumber

        public int getLastImportedItemLineNumber()
        Gets the line number of last imported line at current input.
        Returns:
        current line number of last imported line of input

      • importAll

        public void importAll()
               throws ImpExException
        The import process will be performed from input stream given in constructor. Before calling this method you can configure the DumpHandler, the ErrorHandler and the maximal amount of passes. The call will automatically perform the whole import process including the dumped line processing (if maximal passes are a positive number). You can call the importNext method too for importing item based. This method is only a convenience kind which calls the importNext method until it returns null.
        Throws:
        ImpExException - an error is occurred while import processing (reasoned by input stream errors, dump file handling or item processing)

      • importNext

        public final Item importNext()
                      throws ImpExException
        Tries to import the next item from input stream. If called for one time, you can not call set.. methods anymore. If the input stream ends and there are unresolved lines, the call will automatically start the next pass by using the dump file as new input stream.
        Returns:
        the last imported item
        Throws:
        ImpExException - ImpExException an error is occurred while import processing (reasoned by input stream errors, dump file handling or item processing)

      • activateLogFilter

        protected void activateLogFilter()

      • deactivateLogFilter

        protected void deactivateLogFilter()

      • close

        public void close()
        Closes the used reader instance. Is not needed to call if the import process has finished ( isFinished() returns true).

      • setMaxPass

        public void setMaxPass​(int maxPass)
        Sets the maximal amount of passes the import process will perform. Use -1 for infinit loops (will break if no changes occur anymore).
        Parameters:
        maxPass - new amount of maximal passes

      • setDumpHandler

        public void setDumpHandler​(DumpHandler handler)
        Sets the DumpHandler to use while import process.
        Parameters:
        handler - new dump handler instance

      • setErrorHandler

        public void setErrorHandler​(ErrorHandler handler)
        Sets the ErrorHandler to use while import.
        Parameters:
        handler - new error handler instance

      • setLogFilter

        public void setLogFilter​(ImpExLogFilter filter)
        Sets the logging filter to use while import. The default one extends each message with the current reader location except for info logs.
        Parameters:
        filter - new error handler instance

      • hadError

        public boolean hadError()