Package de.hybris.platform.impex.jalo

Class ImpExReader

  • Direct Known Subclasses:
    BatchingImportReader, ImpExImportReader, ImportBatchParser.BatchingImpexReader, ScriptValidationReader
    public class ImpExReader
extends java.lang.Object
    Base class for reading hybris style csv data. This class is able to parse header and value lines. Comments are ignored and variable definitions are inlined. Bean shell commands will be executed if set. Including of external data (from Media, File, InputStream, CSVReader or from a database) is possible via bean shell commands (java statements starting with #%). Attention! This reader only parses value lines. For interpreting them you need to use an extension like ImpExImportReader.
    The recommended way of using this reader is:
    1. Instantiate a new reader with a constructor giving a source from where the main script is available. This main script should only contain settings via bean shell commands, header definitions and external data inclusions via bean shell commands, no value lines. For available bean shell commands and usage of them see ImpEx Documentation.

      For example:
      ImpExReader reader = new ImpExReader( new CSVReader( "import.impex" ) );

      Example file: "import.impex"
      "#% impex.setRelaxedMode(false);"
      "INSERT_UPDATE Language";"isocode[unique=true]";"name[lang=en]"
      "#% impex.includeExternalData( new CSVReader(""Language.csv"",""windows-1252""));"

      Example file: "Language.csv"
      ;"de";"German"

    2. Configure the reader for your needs, for example:

      reader.enableCodeExecution( true );
      reader.enableExternalImpExSyntaxParsing( true );

    3. Do subsequent calls to readLine() for parsing and getting the headers and value lines.

      Object nextOutput = null;
      do
      {
      nextOutput = reader.readLine();
      if( nextOutput instanceof HeaderDescriptor )
      {
      //do something with header
      }
      if( nextOutput instanceof ValueLine )
      {
      //do something with line
      }
      }
      while(nextOutput != null);

    4. Close the reader.

      reader.close();

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  ImpExReader.ResultSetCSVReader
      A CSVReader extension for reading a SQL ResultSet as CSV.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.Integer FIRST
      The position of current header mode and type within the current line Map.

    • Constructor Summary

      Constructors 
      Constructor Description
      ImpExReader​(CSVReader reader, boolean skipValueLines)
      Instantiates a new reader using the given CSVReader as input.
      ImpExReader​(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode)
      Instantiates a new reader using the given CSVReader as input.
      ImpExReader​(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode, DocumentIDRegistry docIdRegistry)
      Instantiates a new reader using the given CSVReader as input.
      ImpExReader​(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode, DocumentIDRegistry docIdRegistry, InvalidHeaderPolicy headerHandlingPolicy)  
      ImpExReader​(java.io.InputStream input, java.lang.String encoding)
      Instantiates a new reader using the given stream and encoding as input.
      ImpExReader​(java.io.InputStream input, java.lang.String encoding, boolean skipValueLines)
      Instantiates a new reader using the given stream and encoding as input.
      ImpExReader​(java.lang.String fileName, java.lang.String encoding, boolean skipValueLines)
      Instantiates a new reader using the file with given name and encoding as input.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      protected void addDefinition​(java.lang.String cell)
      Adds a definition to the reader context and is therefore available for parsing of headers.
      void addExternalDataMedia​(ImpExMedia ext)
      Adds an external media to the reader context which is then accessible via includeExternalDataMedia.
      void addExternalDataMedias​(java.util.Collection<ImpExMedia> toAdd)
      Adds an collection of external medias to the reader context which are then accessible via includeExternalDataMedia each.
      protected void addHeaderExceptionInfoAsComment​(java.util.Map<java.lang.Integer,​java.lang.String> adjustedLine, HeaderValidationException ex)  
      void addHeaderReplacementRule​(java.lang.String srcPattern, java.lang.String tgtPattern, int prio)  
      protected void addToBeanShellContext​(java.lang.String key, java.lang.Object value)
      Adds a variable referencing to the given object to the bean shell context.
      protected java.util.Map<java.lang.Integer,​java.lang.String> adjustLineIndexes​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Adjusts the line indexes using the column offset set at reader manager.
      protected java.util.List<java.lang.String> applyHeaderReplacements​(java.util.List<java.lang.String> columns)  
      protected void assureBeanShellLoaded()
      Assures that the bean shell instance is loaded.
      protected void checkDefinitonKey​(java.lang.String newKey, java.util.Map currentDefinitions)
      Checks if a key of a definition is a prefix of an already present key.
      void close()
      Closes all used readers.
      protected AbstractCodeLine createCodeLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Interprets the given line as code line.
      protected AbstractCodeLine createCodeLineLegacyWay​(java.util.Map<java.lang.Integer,​java.lang.String> line)  
      protected AbstractCodeLine createCodeLineModernWay​(java.util.Map<java.lang.Integer,​java.lang.String> line)  
      HeaderDescriptor createInvalidHeader​(java.util.Map<java.lang.Integer,​java.lang.String> line, HeaderValidationException ex)  
      protected HeaderDescriptor createNewHeader​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Creates from a header line a header object.
      protected ValueLine createValueLine​(HeaderDescriptor header, java.util.Map<java.lang.Integer,​java.lang.String> line)
      Creates from a line which represents a data row a value line.
      void debug​(java.lang.String msg)
      Debugs a message with the current logger instance.
      void debug​(java.lang.String msg, java.lang.Throwable throwable)
      Debugs a message with the current logger instance.
      void enableCodeExecution​(boolean isOn)
      Sets the code execution flag.
      void enableExternalCodeExecution​(boolean isOn)
      Sets the external code execution flag.
      void enableExternalDataCodeExecution​(boolean isOn)
      Deprecated.
      since ages - useenableExternalCodeExecution(boolean) instead
      void enableExternalImpExSyntaxParsing​(boolean isOn)
      Deprecated.
      since ages - useenableExternalSyntaxParsing(boolean) instead
      void enableExternalSyntaxParsing​(boolean isOn)
      Sets the external syntax parsing flag.
      boolean enterIfBlock​(AbstractCodeLine line)
      Has to be called when an if-condition occurs.
      void error​(java.lang.String msg)
      Prints an error message with the current logger instance.
      void error​(java.lang.String msg, java.lang.Throwable throwable)
      Prints an error message with the current logger instance.
      protected void execute​(AbstractCodeLine line, java.util.Map csvLine)
      Executes the code line at the bean shell.
      protected void execute​(AbstractCodeLine line, java.util.Map csvLine, boolean forEachMode)
      Executes the code line at the bean shell.
      void exitIfBlock()
      Has to be called when an endif-marker occurs.
      ImpExMedia findExternalDataMedia​(java.lang.String code)
      Searches the media with the given code in the context of the reader and returns it.
      protected ImpExMedia findExternalDataMedia​(java.lang.String prefix, java.lang.String code)  
      protected java.lang.String findMarker​(java.lang.String expr)
      Checks if the given expression starts with a marker (special construct, see ImpExConstants.Syntax.CodeMarkers ) and returns the marker string..
      protected java.util.Set<java.lang.String> getAllExternalDataMediaCodes()
      Gets all medias added to reader context for external data inclusion.
      HeaderDescriptor.AttributeConstraintFilter getAttributeConstraintFilter()
      Get the current constraint filter used in HeaderDescriptors created until yet.
      AbstractCodeLine getBeforeEachCode()
      Gets the code line executed after parsing but before interpreting of each line.
      protected java.util.Map<java.lang.Integer,​CSVCellDecorator> getCellDecorators()
      Gets all decorators set for current header.
      CSVReader getCSVReader()
      Gets the reader instance of the main script.
      HeaderDescriptor getCurrentHeader()
      Gets the current header which was set at each read value line.
      java.lang.String getCurrentLocation()
      Gets the current line number of the reader.
      protected CSVReader getCurrentReader()
      Returns the current activated reader used for reading data.
      protected java.util.List<java.lang.Object[]> getDefaultReplacements()  
      DocumentIDRegistry getDocumentIDRegistry()
      Gets the used document ID registry which will be set to each created header.
      java.lang.Object getFromBeanShellContext​(java.lang.String key)
      Gets the value of the given variable within bean shell context.
      InvalidHeaderPolicy getInvalidHeaderPolicy()  
      protected int getLineSize​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Returns the actual length of a value line.
      java.util.Locale getLocale()
      Get the current locale setting used for parsing value lines.
      protected java.util.Map<java.lang.String,​java.lang.Object> getScriptExecutionContext​(java.util.Map<java.lang.Integer,​java.lang.String> line)  
      protected EnumerationValue getStrictMode()  
      EnumerationValue getValidationMode()  
      protected boolean gotInsertedLines()
      Are lines available which have to be used before reading from reader?
      protected boolean hasCellDecorators()
      Checks if there are decorators set for current header.
      void includeExternalData​(CSVReader reader)
      Redirects the main reader to the given temporary reader for including external data.
      void includeExternalData​(CSVReader reader, int columnOffset)
      Redirects the main reader to the given temporary reader for including external data.
      void includeExternalData​(CSVReader reader, int linesToSkip, int columnOffset)
      Redirects the main reader to the given temporary reader for including external data.
      void includeExternalData​(CSVReader reader, int columnOffset, java.lang.String locationText)
      Redirects the main reader to the given temporary reader for including external data.
      void includeExternalData​(java.io.File file, java.lang.String encoding, int linesToSkip)
      Creates a new temporary reader with given file and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalData​(java.io.InputStream inputStream, java.lang.String encoding, boolean setNewHeader)
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalData​(java.io.InputStream inputStream, java.lang.String encoding, char[] delimiter, int linesToSkip, int columnOffset)
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalData​(java.io.InputStream inputStream, java.lang.String encoding, int linesToSkip)
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalData​(java.io.InputStream inputStream, java.lang.String encoding, int linesToSkip, int columnOffset)
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalData​(java.lang.String fileName, java.lang.String fileEncoding, int linesToSkip)
      Creates a new temporary reader with file of given filename and encoding and redirects the main reader to that temporary reader for including external data.
      void includeExternalDataMedia​(ImpExMedia media)
      Redirects the import reader to external data stored inside a media.
      void includeExternalDataMedia​(ImpExMedia media, int columnOffset)
      Redirects the import reader to external data stored inside a media.
      void includeExternalDataMedia​(ImpExMedia media, java.lang.String encoding, char delimiter, int linesToSkip, int columnOffset)
      Redirects the import reader to external data stored inside a media.
      ATTENTION: The given parameters will be used for configuring of media, not the attributes set at media.
      void includeExternalDataMedia​(java.lang.String code)
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
      void includeExternalDataMedia​(java.lang.String code, int columnOffset)
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
      void includeExternalDataMedia​(java.lang.String code, int linesToSkip, int columnOffset)
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      void includeExternalDataMedia​(java.lang.String code, java.lang.String encoding)
      Deprecated.
      since ages - useincludeExternalDataMedia(String) instead
      void includeExternalDataMedia​(java.lang.String code, java.lang.String encoding, char delimiter, int linesToSkip, int columnOffset)
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      void includeExternalDataMedia​(java.lang.String code, java.lang.String encoding, int columnOffset)
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      void includeExternalDataMedia​(java.lang.String code, java.lang.String encoding, int linesToSkip, int columnOffset)
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      void includeSQLData​(java.lang.String sqlStatement)
      Uses current settings to establish a database connection, to fire given statement and to parse the result set to value lines.
      void includeSQLData​(java.lang.String sqlStatement, int skip, int offset)
      Fires the given statement to the database connection established with current settings and parses the lines of the resulting result set as value lines.
      void includeSQLData​(java.lang.String url, java.lang.String user, java.lang.String password, java.lang.String className, java.lang.String sqlStatement)
      Establishes a connection to a database with the given parameters, fires the given statement and includes the resulting result set as external data.
      void info​(java.lang.String msg)
      Prints an info message with the current logger instance.
      void info​(java.lang.String msg, java.lang.Throwable throwable)
      Prints an info message with the current logger instance.
      void initDatabase​(java.lang.String url, java.lang.String user, java.lang.String password, java.lang.String className)
      Initializes the parameters for using a database connection using includeSQLData.
      void insertLine​(java.util.Map csvLine)
      Inserts the line into reading process.
      boolean isCodeExecutionEnabled()
      Is parsing and interpreting of code lines in data enabled?
      protected boolean isCodeLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given line is a code line (introduced with #%).
      boolean isCombinedSearchEnabled()
      Tells whether or not to use combined search queries.
      protected boolean isCommentLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given line is a comment (introduced with #).
      boolean isDebugEnabled()  
      protected boolean isDefinition​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given line declares a definition.
      protected boolean isEmptyLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if line has no columns.
      protected boolean isEndUserRights​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given ends a UserRight section (consists only of $END_USERRIGHTS).
      boolean isExternalCodeExecutionEnabled()
      Is parsing and interpreting of code lines in external data enabled?
      boolean isExternalSyntaxParsingEnabled()
      Will non value lines in external lines be interpreted or parsed as value lines.
      protected boolean isHeaderLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given line is a header line (introduced with header mode string).
      boolean isIncludingExternalData()
      Tells whether or not this reader fetches its data from a external source currently (see includeExternalData(CSVReader) etc.
      boolean isInfoEnabled()  
      protected boolean isNotInInactiveIfBlock​(AbstractCodeLine curLine)
      Tests if you are allowed to execute code, based on the current if-block situation.
      protected boolean isStartUserRights​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks if the given introduces a UserRight section (consists only of $START_USERRIGHTS).
      protected void legacyExecute​(AbstractCodeLine line, java.util.Map<java.lang.Integer,​java.lang.String> csvLine)  
      protected java.util.List<java.lang.String> lineToList​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Rearranges a csv line map into a list object.
      protected void modernExecute​(AbstractScriptingEngineCodeLine line, java.util.Map csvLine)  
      static HeaderDescriptor parseHeader​(java.lang.String headerLine)
      Generates a header object from given line.
      static HeaderDescriptor parseHeader​(java.lang.String headerLine, DocumentIDRegistry docIdRegistry)
      Generates a header object from given line.
      static HeaderDescriptor parseHeader​(java.lang.String headerLine, EnumerationValue mode)
      Generates a header object from given line.
      static HeaderDescriptor parseHeader​(java.lang.String headerLine, EnumerationValue mode, DocumentIDRegistry docIdRegistry)
      Generates a header object from given line.
      protected void processCodeLine​(AbstractCodeLine line)
      Processes given code line.
      protected boolean processMarkerCodeLine​(AbstractCodeLine line)
      Processes the special logic of the marker set to the given code line and returns true, if the code line should be executed or the marker prohibits the execution.
      java.lang.Object readLine()
      Fetches the next line, parses it and returns it encapsulated in a HeaderDescriptor if the line was a header definition or a ValueLine if it was a data row.
      protected java.util.Map<java.lang.Integer,​java.lang.String> readNextCSVLine()
      Reads the next line.
      void removeExternalDataMedia​(Media ext)
      Removes an media from the reader context and is therefore not includable as external data anymore.
      void removeExternalDataMedias​(java.util.Collection<Media> toRemove)
      Removes an collection medias from the reader context and are therefore not integrable as external data anymore.
      protected java.util.Map<java.lang.Integer,​java.lang.String> replaceDefinitions​(java.util.Map<java.lang.Integer,​java.lang.String> line)
      Checks the given line for definition usage and replace all definition keys with the correct definition values.
      void setAttributeConstraintFilter​(HeaderDescriptor.AttributeConstraintFilter filter)
      Sets a new constraint filter (for defining methods like isMandatory for attributes) used in each created HeaderDescriptor from now.
      protected void setBeanShellContext​(bsh.Interpreter shell, java.util.Map<java.lang.Integer,​java.lang.String> line)
      Declares default variables within the bean shell context which therefore are available within bean shell code interpretation.
      protected void setBeforeEachCode​(AbstractCodeLine line)
      Sets the code line executed after parsing but before interpreting of each line.
      protected void setCellDecorators​(HeaderDescriptor header)
      Extracts all decorators from given header columns and stores them for line parsing.
      void setCurrentHeader​(HeaderDescriptor header)
      Sets the given header instance as current header.
      void setCurrentHeader​(java.lang.String headerCSVDefinition)
      Generates from given header line a HeaderDescriptor and sets it as current header for further reading of input.
      protected void setInvalidHeaderPolicy​(InvalidHeaderPolicy invalidHeaderPolicy)  
      void setLocale​(java.util.Locale locale)
      Sets the locale used while parsing value lines.
      void setRelaxedMode​(boolean enable)
      Deprecated.
      since 3.0, use setValidationMode(EnumerationValue) instead
      void setRelaxedMode​(java.lang.String enable)
      Deprecated.
      since 3.0, use setValidationMode(EnumerationValue) instead
      void setValidationMode​(EnumerationValue headerValidationMode)
      Sets the header valdiation mode used for all created headers.
      void setValidationMode​(java.lang.String mode)  
      protected void sortRulesList​(java.util.List<java.lang.Object[]> rulesList)  
      protected java.lang.String[] splitDefinitonCell​(java.lang.String cell)
      Splits the given definition in key and value pair.
      protected void storeUserRightsLine​(java.util.Map line)
      Concatenates given line and adds it to the already read user rights lines for further special processing.
      void warn​(java.lang.String msg)
      Prints an warn message with the current logger instance.
      void warn​(java.lang.String msg, java.lang.Throwable throwable)
      Prints an warn message with the current logger instance.
      protected void writeUserRightsLines()
      Writes all stored UserRights lines to an temporary buffer and imports this file with the ImportExportUserRightsHelper.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • FIRST

        public static final java.lang.Integer FIRST
        The position of current header mode and type within the current line Map.

    • Constructor Detail

      • ImpExReader

        public ImpExReader​(java.lang.String fileName,
                   java.lang.String encoding,
                   boolean skipValueLines)
            throws java.io.UnsupportedEncodingException,
                   java.io.FileNotFoundException
        Instantiates a new reader using the file with given name and encoding as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
        Parameters:
        fileName - the name of the file used as input for the reader
        encoding - used encoding for reading the file
        skipValueLines - if true, all value lines (lines within input containing data) will be not interpreted and therefore not returned by call of readLine()
        Throws:
        java.io.UnsupportedEncodingException - the given encoding is unknown
        java.io.FileNotFoundException - there is no file with the given file name

      • ImpExReader

        public ImpExReader​(java.io.InputStream input,
                   java.lang.String encoding)
            throws java.io.UnsupportedEncodingException
        Instantiates a new reader using the given stream and encoding as input. Skipping of value lines will be not enabled, relaxed mode is disabled and a new document ID registry will be instantiated.
        Parameters:
        input - stream from which the script is read
        encoding - used encoding for reading from stream
        Throws:
        java.io.UnsupportedEncodingException - encoding is unknown

      • ImpExReader

        public ImpExReader​(java.io.InputStream input,
                   java.lang.String encoding,
                   boolean skipValueLines)
            throws java.io.UnsupportedEncodingException
        Instantiates a new reader using the given stream and encoding as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
        Parameters:
        input - stream from which the script is read
        encoding - used encoding for reading from stream
        skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
        Throws:
        java.io.UnsupportedEncodingException - encoding is unknown

      • ImpExReader

        public ImpExReader​(CSVReader reader,
                   boolean skipValueLines)
        Instantiates a new reader using the given CSVReader as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
        Parameters:
        reader - the reader from which the script is read
        skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them

      • ImpExReader

        public ImpExReader​(CSVReader reader,
                   boolean skipValueLines,
                   EnumerationValue headerValidationMode)
        Instantiates a new reader using the given CSVReader as input. A new document ID registry will be instantiated.
        Parameters:
        reader - the reader from which the script is read
        skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
        headerValidationMode - the following modes are supported:
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT

      • ImpExReader

        public ImpExReader​(CSVReader reader,
                   boolean skipValueLines,
                   EnumerationValue headerValidationMode,
                   DocumentIDRegistry docIdRegistry)
        Instantiates a new reader using the given CSVReader as input.
        Parameters:
        reader - the reader from which the script is read
        skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
        headerValidationMode - the following modes are supported:
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT
        • docIdRegistry - the used document ID registry set to each created ImpExHeaderDesriptor

    • Method Detail

      • setInvalidHeaderPolicy

        protected void setInvalidHeaderPolicy​(InvalidHeaderPolicy invalidHeaderPolicy)

      • getDefaultReplacements

        protected java.util.List<java.lang.Object[]> getDefaultReplacements()

      • sortRulesList

        protected void sortRulesList​(java.util.List<java.lang.Object[]> rulesList)

      • addHeaderReplacementRule

        public void addHeaderReplacementRule​(java.lang.String srcPattern,
                                     java.lang.String tgtPattern,
                                     int prio)

      • setAttributeConstraintFilter

        public void setAttributeConstraintFilter​(HeaderDescriptor.AttributeConstraintFilter filter)
        Sets a new constraint filter (for defining methods like isMandatory for attributes) used in each created HeaderDescriptor from now.
        Parameters:
        filter - the new filter used in headers

      • getAttributeConstraintFilter

        public HeaderDescriptor.AttributeConstraintFilter getAttributeConstraintFilter()
        Get the current constraint filter used in HeaderDescriptors created until yet.
        Returns:
        the current constraint filter

      • parseHeader

        public static HeaderDescriptor parseHeader​(java.lang.String headerLine)
                                    throws ImpExException
        Generates a header object from given line. Relaxed mode is set to false and a new document ID registry is created.
        Parameters:
        headerLine - the line which will be parsed and a header object is created from
        Returns:
        a header object representing the given line
        Throws:
        ImpExException - the given line does not represent a header or is misspelled

      • parseHeader

        public static HeaderDescriptor parseHeader​(java.lang.String headerLine,
                                           DocumentIDRegistry docIdRegistry)
                                    throws ImpExException
        Generates a header object from given line. Relaxed mode is set to false.
        Parameters:
        headerLine - the line which will be parsed and a header object is created from
        docIdRegistry - a document ID registry which will be attached to the header object
        Returns:
        a header object representing the given line
        Throws:
        ImpExException - the given line does not represent a header or is misspelled

      • parseHeader

        public static HeaderDescriptor parseHeader​(java.lang.String headerLine,
                                           EnumerationValue mode)
                                    throws ImpExException
        Generates a header object from given line. A new document ID registry is created.
        Parameters:
        headerLine - the line which will be parsed and a header object is created from
        mode - should the relaxed mode be set in header object
        Returns:
        a header object representing the given line
        Throws:
        ImpExException - the given line does not represent a header or is misspelled

      • parseHeader

        public static HeaderDescriptor parseHeader​(java.lang.String headerLine,
                                           EnumerationValue mode,
                                           DocumentIDRegistry docIdRegistry)
                                    throws ImpExException
        Generates a header object from given line.
        Parameters:
        headerLine - the line which will be parsed and a header object is created from
        mode - should the relaxed mode be set in header object
        docIdRegistry - a document ID registry which will be attached to the header object
        Returns:
        a header object representing the given line
        Throws:
        ImpExException - the given line does not represent a header or is misspelled

      • isExternalSyntaxParsingEnabled

        public boolean isExternalSyntaxParsingEnabled()
        Will non value lines in external lines be interpreted or parsed as value lines.
        Returns:
        is external syntax parsing activated?

      • enableExternalImpExSyntaxParsing

        @Deprecated
public void enableExternalImpExSyntaxParsing​(boolean isOn)
        Deprecated.
        since ages - useenableExternalSyntaxParsing(boolean) instead
        Sets the external syntax parsing flag. If is not set, syntax in external data like headers can not be interpreted.
        Parameters:
        isOn - set true if external parsing should be activated

      • enableExternalSyntaxParsing

        public void enableExternalSyntaxParsing​(boolean isOn)
        Sets the external syntax parsing flag. If is not set, syntax in external data like headers can not be interpreted.
        Parameters:
        isOn - set true if external parsing should be activated

      • isCombinedSearchEnabled

        public boolean isCombinedSearchEnabled()
        Tells whether or not to use combined search queries. This means that all item reference expressions of a single line are queried using *one* big UNION query instead of on query per single reference expression. This avoids database round trips and possibly safes cpu in database itself.
        Returns:
        true if config parameter

      • setLocale

        public void setLocale​(java.util.Locale locale)
        Sets the locale used while parsing value lines. Locale is important in case of numbers and dates.
        Parameters:
        locale - the new locale used from now

      • getLocale

        public java.util.Locale getLocale()
        Get the current locale setting used for parsing value lines.
        Returns:
        current locale setting

      • close

        public void close()
           throws java.io.IOException
        Closes all used readers.
        Throws:
        java.io.IOException - error while closing streams

      • getCSVReader

        public CSVReader getCSVReader()
        Gets the reader instance of the main script. To get the current reader call getCurrentReader().
        Returns:
        main reader

      • addExternalDataMedia

        public void addExternalDataMedia​(ImpExMedia ext)
        Adds an external media to the reader context which is then accessible via includeExternalDataMedia.
        Parameters:
        ext - the new media which can be included as external data source now

      • addExternalDataMedias

        public void addExternalDataMedias​(java.util.Collection<ImpExMedia> toAdd)
        Adds an collection of external medias to the reader context which are then accessible via includeExternalDataMedia each.
        Parameters:
        toAdd - the new medias which can be included as external data source now

      • removeExternalDataMedia

        public void removeExternalDataMedia​(Media ext)
        Removes an media from the reader context and is therefore not includable as external data anymore.
        Parameters:
        ext - the media which will be removed from context

      • removeExternalDataMedias

        public void removeExternalDataMedias​(java.util.Collection<Media> toRemove)
        Removes an collection medias from the reader context and are therefore not integrable as external data anymore.
        Parameters:
        toRemove - the media collection which will be removed from context

      • getAllExternalDataMediaCodes

        protected java.util.Set<java.lang.String> getAllExternalDataMediaCodes()
        Gets all medias added to reader context for external data inclusion.
        Returns:
        set of all integrable medias

      • findExternalDataMedia

        public ImpExMedia findExternalDataMedia​(java.lang.String code)
                                 throws JaloBusinessException
        Searches the media with the given code in the context of the reader and returns it.
        Parameters:
        code - code of the media
        Returns:
        media with given code or null if no data is found
        Throws:
        JaloBusinessException - a media with the code was found but has had no data attached

      • findExternalDataMedia

        protected ImpExMedia findExternalDataMedia​(java.lang.String prefix,
                                           java.lang.String code)

      • isIncludingExternalData

        public boolean isIncludingExternalData()
        Tells whether or not this reader fetches its data from a external source currently (see includeExternalData(CSVReader) etc. ).
        Returns:
        are we are reading external data at the moment

      • setCurrentHeader

        public void setCurrentHeader​(java.lang.String headerCSVDefinition)
                      throws ImpExException
        Generates from given header line a HeaderDescriptor and sets it as current header for further reading of input.
        Parameters:
        headerCSVDefinition - line defining a header
        Throws:
        ImpExException - error while generation of the header object

      • setCurrentHeader

        public void setCurrentHeader​(HeaderDescriptor header)
        Sets the given header instance as current header. The current header is set for each read value line.
        Parameters:
        header - the header which will be used from now as header for read value lines

      • getCurrentHeader

        public HeaderDescriptor getCurrentHeader()
        Gets the current header which was set at each read value line.
        Returns:
        current header instance, or null if no header is activated (at start of reading)

      • initDatabase

        public final void initDatabase​(java.lang.String url,
                               java.lang.String user,
                               java.lang.String password,
                               java.lang.String className)
        Initializes the parameters for using a database connection using includeSQLData.
        Parameters:
        url - url to the database
        user - name of user / account in database
        password - password of user
        className - name of database driver (JDBC)

      • includeSQLData

        public void includeSQLData​(java.lang.String url,
                           java.lang.String user,
                           java.lang.String password,
                           java.lang.String className,
                           java.lang.String sqlStatement)
        Establishes a connection to a database with the given parameters, fires the given statement and includes the resulting result set as external data. Each line of the result set will be parsed as value line.

        Example script:
        #% impex.enableCodeExecution( true );
        #% impex.setRelaxedMode( true );
        INSERT_UPDATE Product; code[unique=true]
        #% impex.includeSQLData("jdbc:mysql://localhost/testdb?user=hendrik&password=hendrik" ,"com.mysql.jdbc.Driver","Select * from Products");

        Parameters:
        url - url to the database
        user - name of the user / account at database
        password - password of user
        className - name of the database driver
        sqlStatement - error while establishing connection or reading result set

      • includeSQLData

        public void includeSQLData​(java.lang.String sqlStatement)
        Uses current settings to establish a database connection, to fire given statement and to parse the result set to value lines.
        Parameters:
        sqlStatement - error while accessing the result set of given statement

      • includeSQLData

        public void includeSQLData​(java.lang.String sqlStatement,
                           int skip,
                           int offset)
        Fires the given statement to the database connection established with current settings and parses the lines of the resulting result set as value lines.

        Example script:
        #% impex.enableCodeExecution( true );
        #% impex.setRelaxedMode( true );
        #% impex.initDatabase( "jdbc:mysql://localhost/testdb?user=hendrik&password=hendrik","com.mysql.jdbc.Driver" );
        INSERT_UPDATE Product; code[unique=true]
        #% impex.includeSQLData("Select * from Products");

        Parameters:
        sqlStatement - statement which will be executed
        skip - lines to skip within result set
        offset - amount of columns to skip from left

      • includeExternalData

        public void includeExternalData​(java.io.File file,
                                java.lang.String encoding,
                                int linesToSkip)
                         throws java.io.UnsupportedEncodingException,
                                java.io.FileNotFoundException
        Creates a new temporary reader with given file and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. Reading from temporary reader can be stopped by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
        Parameters:
        file - file which contains external data to be included
        encoding - the encoding used in given file
        linesToSkip - amount of lines the reader of the file will initial skip
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown
        java.io.FileNotFoundException - given file is not found

      • includeExternalData

        public void includeExternalData​(java.io.InputStream inputStream,
                                java.lang.String encoding,
                                int linesToSkip)
                         throws java.io.UnsupportedEncodingException
        Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
        Parameters:
        inputStream - input stream from which external data to be included is accessable
        encoding - the used encoding
        linesToSkip - amount of lines the reader of the stream will initial skip
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown

      • includeExternalData

        public void includeExternalData​(java.io.InputStream inputStream,
                                java.lang.String encoding,
                                int linesToSkip,
                                int columnOffset)
                         throws java.io.UnsupportedEncodingException
        Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow.
        Parameters:
        inputStream - input stream from which external data to be included is accessible
        encoding - the used encoding
        linesToSkip - amount of lines the reader of the stream will initial skip
        columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because the data columns do not have a first column with type information. So the header columns have to be shifted one to the left =>-1.
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown

      • includeExternalData

        public void includeExternalData​(java.io.InputStream inputStream,
                                java.lang.String encoding,
                                char[] delimiter,
                                int linesToSkip,
                                int columnOffset)
                         throws java.io.UnsupportedEncodingException
        Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow.
        Parameters:
        inputStream - input stream from which external data to be included is accessible
        encoding - the used encoding
        delimiter - field separators used in external data
        linesToSkip - amount of lines the reader of the stream will initial skip
        columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because the data columns do not have a first column with type information. So the header columns have to be shifted one to the left =>-1.
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown

      • includeExternalData

        public void includeExternalData​(java.lang.String fileName,
                                java.lang.String fileEncoding,
                                int linesToSkip)
                         throws java.io.UnsupportedEncodingException,
                                java.io.FileNotFoundException
        Creates a new temporary reader with file of given filename and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. Reading from temporary reader will stop by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
        Parameters:
        fileName - file which contains external data to be included
        fileEncoding - the encoding used in given file
        linesToSkip - amount of lines the reader of the file will initial skip
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown
        java.io.FileNotFoundException - given file is not found

      • includeExternalData

        public void includeExternalData​(CSVReader reader)
        Redirects the main reader to the given temporary reader for including external data. So you can include external data given with the reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
        Parameters:
        reader - the temporary reader which reads from external data

      • includeExternalData

        public void includeExternalData​(CSVReader reader,
                                int linesToSkip,
                                int columnOffset)
        Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
        Parameters:
        reader - the temporary reader which reads from external data
        linesToSkip - amount of lines which will be skipped when start reading from external data
        columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.

      • includeExternalData

        public void includeExternalData​(CSVReader reader,
                                int columnOffset)
        Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
        Parameters:
        reader - the temporary reader which reads from external data
        columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.

      • includeExternalData

        public void includeExternalData​(CSVReader reader,
                                int columnOffset,
                                java.lang.String locationText)
        Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
        Parameters:
        reader - the temporary reader which reads from external data
        columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.
        locationText - identifier of the external data for logging purposes

      • includeExternalData

        public void includeExternalData​(java.io.InputStream inputStream,
                                java.lang.String encoding,
                                boolean setNewHeader)
                         throws java.io.UnsupportedEncodingException,
                                ImpExException
        Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
        Parameters:
        inputStream - input stream from which external data to be included is accessible
        encoding - the used encoding
        setNewHeader - if true, the first line of the external data will be tried to interpreted as header
        Throws:
        java.io.UnsupportedEncodingException - given encoding is unknown
        ImpExException - error while parsing header from first line (only when setNewHeader==true)

      • includeExternalDataMedia

        public void includeExternalDataMedia​(java.lang.String code)
                              throws java.io.UnsupportedEncodingException
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped and column offset is set to -1 (data is not in hybris style).
        Parameters:
        code - the media code
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        public void includeExternalDataMedia​(java.lang.String code,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped.
        Parameters:
        code - the media code
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        @Deprecated
public void includeExternalDataMedia​(java.lang.String code,
                                     int linesToSkip,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Deprecated.
        since ages - useincludeExternalDataMedia(String, int) instead
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
        The given linesToSkip value will be used instead of the equivalent attribute set at the media. It is preferred to set the encoding and linesToSkip directly at the media and to use includeExternalDataMedia(String, int) instead.
        Parameters:
        code - the media code
        linesToSkip - amount of lines which will be skipped when start reading from external data
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        @Deprecated
public void includeExternalDataMedia​(java.lang.String code,
                                     java.lang.String encoding)
                              throws java.io.UnsupportedEncodingException
        Deprecated.
        since ages - useincludeExternalDataMedia(String) instead
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped and column offset is set to -1 (data is not in hybris style).
        The given encoding will be used instead of the encoding set at the media. It is preferred to set the encoding directly at the media and to use includeExternalDataMedia(String) instead.
        Parameters:
        code - the media code
        encoding - the encoding of the csv data stored inside the media
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        @Deprecated
public void includeExternalDataMedia​(java.lang.String code,
                                     java.lang.String encoding,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Deprecated.
        since ages - useincludeExternalDataMedia(String, int) instead
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped.
        The given encoding will be used instead of the encoding set at the media. It is preferred to set the encoding directly at the media and to use includeExternalDataMedia(String) instead.
        Parameters:
        code - the media code
        encoding - the encoding of the csv data stored inside the media
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        @Deprecated
public void includeExternalDataMedia​(java.lang.String code,
                                     java.lang.String encoding,
                                     int linesToSkip,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Deprecated.
        since ages - useincludeExternalDataMedia(String, int) instead
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
        The given encoding and linesToSkip value will be used instead of the equivalent attribute set at the media. It is preferred to set the encoding and linesToSkip directly at the media and to use includeExternalDataMedia(String) instead.
        Parameters:
        code - the media code
        encoding - the encoding of the csv data stored inside the media
        linesToSkip - amount of lines which will be skipped when start reading from external data
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        @Deprecated
public void includeExternalDataMedia​(java.lang.String code,
                                     java.lang.String encoding,
                                     char delimiter,
                                     int linesToSkip,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Deprecated.
        since ages - useincludeExternalDataMedia(String, int) instead
        Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
        The given encoding,delimiter and linesToSkip values will be used instead of the equivalent attribute set at the media. It is preferred to set these values directly at the media and to use includeExternalDataMedia(String) instead.
        Parameters:
        code - the media code
        encoding - the encoding of the csv data stored inside the media
        delimiter - field separator used in external data
        linesToSkip - amount of lines which will be skipped when start reading from external data
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        public void includeExternalDataMedia​(ImpExMedia media)
                              throws java.io.UnsupportedEncodingException
        Redirects the import reader to external data stored inside a media. ColumnOffset will be set to -1
        Parameters:
        media - the media containing the external data
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        public void includeExternalDataMedia​(ImpExMedia media,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Redirects the import reader to external data stored inside a media.
        Parameters:
        media - the media containing the external data
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • includeExternalDataMedia

        public void includeExternalDataMedia​(ImpExMedia media,
                                     java.lang.String encoding,
                                     char delimiter,
                                     int linesToSkip,
                                     int columnOffset)
                              throws java.io.UnsupportedEncodingException
        Redirects the import reader to external data stored inside a media.
        ATTENTION: The given parameters will be used for configuring of media, not the attributes set at media. So if you have set the encoding attribute at the media it will be ignored and the given encoding will be used instead.
        Parameters:
        media - media instance holding the external data
        encoding - the encoding of the csv data stored inside the media
        delimiter - field separator used in external data
        linesToSkip - amount of lines which will be skipped when start reading from external data
        columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
        Throws:
        java.io.UnsupportedEncodingException - unknown encoding given

      • getCurrentLocation

        public java.lang.String getCurrentLocation()
        Gets the current line number of the reader. If the current reader is temporary (reads external data) the location in external data is appended too.
        Returns:
        current line number of reader

      • getCurrentReader

        protected CSVReader getCurrentReader()
        Returns the current activated reader used for reading data. It will return the main reader or the current temporary reader if external data is read.
        Returns:
        currently active reader

      • getDocumentIDRegistry

        public DocumentIDRegistry getDocumentIDRegistry()
        Gets the used document ID registry which will be set to each created header.
        Returns:
        current registry

      • gotInsertedLines

        protected boolean gotInsertedLines()
        Are lines available which have to be used before reading from reader?
        Returns:
        true if inserted lines are available

      • insertLine

        public void insertLine​(java.util.Map csvLine)
        Inserts the line into reading process. The line will be processed before reading next line from reader.
        Parameters:
        csvLine - line which will be inserted in line processing

      • readNextCSVLine

        protected java.util.Map<java.lang.Integer,​java.lang.String> readNextCSVLine()
        Reads the next line. It will be read from buffer with inserted lines or when empty from active reader. The line is a map consisting of each read unescaped column field (determined by field separator set at the used reader) mapped to the column position in associated header.
        Returns:
        next read line

      • adjustLineIndexes

        protected java.util.Map<java.lang.Integer,​java.lang.String> adjustLineIndexes​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Adjusts the line indexes using the column offset set at reader manager. The key values will be summed with the set column offset.
        Parameters:
        line - line to adjust
        Returns:
        adjusted line

      • readLine

        public java.lang.Object readLine()
                          throws ImpExException
        Fetches the next line, parses it and returns it encapsulated in a HeaderDescriptor if the line was a header definition or a ValueLine if it was a data row.
        Returns:
        either HeaderDescriptor or ValueLine instances or null if end of data is reached.
        Throws:
        ImpExException - if a header could not be parsed properly

      • addHeaderExceptionInfoAsComment

        protected void addHeaderExceptionInfoAsComment​(java.util.Map<java.lang.Integer,​java.lang.String> adjustedLine,
                                               HeaderValidationException ex)

      • hasCellDecorators

        protected boolean hasCellDecorators()
        Checks if there are decorators set for current header.
        Returns:
        true if decorators are set

      • getCellDecorators

        protected java.util.Map<java.lang.Integer,​CSVCellDecorator> getCellDecorators()
        Gets all decorators set for current header.
        Returns:
        decorators set to current header

      • setCellDecorators

        protected void setCellDecorators​(HeaderDescriptor header)
        Extracts all decorators from given header columns and stores them for line parsing.
        Parameters:
        header - header from which the decorators will be extracted

      • storeUserRightsLine

        protected void storeUserRightsLine​(java.util.Map line)
        Concatenates given line and adds it to the already read user rights lines for further special processing.
        Parameters:
        line - line with a UserRight data row

      • addDefinition

        protected void addDefinition​(java.lang.String cell)
        Adds a definition to the reader context and is therefore available for parsing of headers. Is a definition with same key already present it will be replaced by this one.
        Parameters:
        cell - string containing the definition

      • splitDefinitonCell

        protected java.lang.String[] splitDefinitonCell​(java.lang.String cell)
        Splits the given definition in key and value pair.
        Parameters:
        cell - the definition to split
        Returns:
        array consisting of two fields key and value: String[{key},{value}]

      • checkDefinitonKey

        protected void checkDefinitonKey​(java.lang.String newKey,
                                 java.util.Map currentDefinitions)
                          throws ImpExException
        Checks if a key of a definition is a prefix of an already present key. If no clash exist, the method will pass, else throws an exception.
        Parameters:
        newKey - key to check
        currentDefinitions - map with present definition
        Throws:
        ImpExException - the given key is a prefix of a present key

      • replaceDefinitions

        protected java.util.Map<java.lang.Integer,​java.lang.String> replaceDefinitions​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks the given line for definition usage and replace all definition keys with the correct definition values.
        Parameters:
        line - line to check
        Returns:
        line with replaced definition keys

      • findMarker

        protected java.lang.String findMarker​(java.lang.String expr)
        Checks if the given expression starts with a marker (special construct, see ImpExConstants.Syntax.CodeMarkers ) and returns the marker string..
        Parameters:
        expr - the expression to check
        Returns:
        the marker string if the expression starts with it or null

      • createCodeLine

        protected AbstractCodeLine createCodeLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Interprets the given line as code line. Therefore, it cuts code declaration (#%) in first column, cuts and remembers defined markers and constructs a code line object. It will return a code line object always, even there is no code declaration, so you have to check first using isCodeLine(Map).
        Parameters:
        line - line to interpret
        Returns:
        code line instance representing this line

      • createCodeLineLegacyWay

        protected AbstractCodeLine createCodeLineLegacyWay​(java.util.Map<java.lang.Integer,​java.lang.String> line)

      • createCodeLineModernWay

        protected AbstractCodeLine createCodeLineModernWay​(java.util.Map<java.lang.Integer,​java.lang.String> line)

      • createNewHeader

        protected HeaderDescriptor createNewHeader​(java.util.Map<java.lang.Integer,​java.lang.String> line)
                                    throws HeaderValidationException
        Creates from a header line a header object. It is not checked, if the line describes a header.
        Parameters:
        line - line for which a header object is needed
        Returns:
        created header object
        Throws:
        HeaderValidationException - header is not valid

      • applyHeaderReplacements

        protected java.util.List<java.lang.String> applyHeaderReplacements​(java.util.List<java.lang.String> columns)

      • getLineSize

        protected int getLineSize​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Returns the actual length of a value line. Since a line contains map entries only not all cell must be filled so line.size() is not sufficient for that!
        Parameters:
        line - line to check
        Returns:
        index of last column in line (column with largest index)

      • lineToList

        protected java.util.List<java.lang.String> lineToList​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Rearranges a csv line map into a list object. Cell values are copied into a new list where each column with index x will be placed in list bucket x.
        Parameters:
        line - line to transform
        Returns:
        list of columns with null entries for non defined indexes

      • isEmptyLine

        protected boolean isEmptyLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if line has no columns.
        Parameters:
        line - line to check
        Returns:
        true, if line has no columns

      • setValidationMode

        public void setValidationMode​(java.lang.String mode)

      • setValidationMode

        public void setValidationMode​(EnumerationValue headerValidationMode)
        Sets the header valdiation mode used for all created headers.
        Parameters:
        headerValidationMode - the following modes are supported:
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
      • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT

      • isCommentLine

        protected boolean isCommentLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given line is a comment (introduced with #).
        Parameters:
        line - line to check
        Returns:
        true, if line is a comment (starts with #)

      • isCodeLine

        protected boolean isCodeLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given line is a code line (introduced with #%).
        Parameters:
        line - line to check
        Returns:
        true, if line is a code line (starts with #%)

      • isHeaderLine

        protected boolean isHeaderLine​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given line is a header line (introduced with header mode string).
        Parameters:
        line - line to check
        Returns:
        true, if line is a header line

      • isDefinition

        protected boolean isDefinition​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given line declares a definition.
        Parameters:
        line - line to check
        Returns:
        true, if line declares a definition

      • isStartUserRights

        protected boolean isStartUserRights​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given introduces a UserRight section (consists only of $START_USERRIGHTS).
        Parameters:
        line - line to check
        Returns:
        true, if line introduces a UserRight section

      • isEndUserRights

        protected boolean isEndUserRights​(java.util.Map<java.lang.Integer,​java.lang.String> line)
        Checks if the given ends a UserRight section (consists only of $END_USERRIGHTS).
        Parameters:
        line - line to check
        Returns:
        true, if line ends a UserRight section

      • createValueLine

        protected ValueLine createValueLine​(HeaderDescriptor header,
                                    java.util.Map<java.lang.Integer,​java.lang.String> line)
        Creates from a line which represents a data row a value line.
        Parameters:
        header - the header which corresponds to the given line
        line - line for which a value line is needed
        Returns:
        value line created from given line

      • isCodeExecutionEnabled

        public final boolean isCodeExecutionEnabled()
        Is parsing and interpreting of code lines in data enabled?
        Returns:
        code parsing is enabled

      • enableCodeExecution

        public void enableCodeExecution​(boolean isOn)
        Sets the code execution flag. Is code execution enabled, the parsing and interpreting of code lines is enabled. Otherwise these lines will be seen as comments.
        Parameters:
        isOn - Should code execution be enabled?

      • isExternalCodeExecutionEnabled

        public final boolean isExternalCodeExecutionEnabled()
        Is parsing and interpreting of code lines in external data enabled?
        Returns:
        code parsing and interpreting is enabled

      • enableExternalDataCodeExecution

        @Deprecated
public void enableExternalDataCodeExecution​(boolean isOn)
        Deprecated.
        since ages - useenableExternalCodeExecution(boolean) instead
        Sets the external code execution flag. Is external code execution enabled, the parsing and interpreting of code lines in external data is enabled. Otherwise these lines will be seen as comments.
        Parameters:
        isOn - Should external code execution be enabled?

      • enableExternalCodeExecution

        public void enableExternalCodeExecution​(boolean isOn)
        Sets the external code execution flag. Is external code execution enabled, the parsing and interpreting of code lines in external data is enabled. Otherwise these lines will be seen as comments.
        Parameters:
        isOn - Should external code execution be enabled?

      • setBeforeEachCode

        protected void setBeforeEachCode​(AbstractCodeLine line)
        Sets the code line executed after parsing but before interpreting of each line.
        Parameters:
        line - code line to remember for before execution

      • getBeforeEachCode

        public AbstractCodeLine getBeforeEachCode()
        Gets the code line executed after parsing but before interpreting of each line.
        Returns:
        code line of beforeEach marker

      • enterIfBlock

        public boolean enterIfBlock​(AbstractCodeLine line)
        Has to be called when an if-condition occurs. Puts the current code line onto the if-stack. If there is already an 'if' on the stack and is this if is inactive (related if-condition was evaluated false) then the result was the current if-condition is also set to false ignoring the real result, because an if-block within an inactive if-block can must not be executed.
        Parameters:
        line - the current code line containing the if-condition

      • exitIfBlock

        public void exitIfBlock()
        Has to be called when an endif-marker occurs. Pops the current if code line from the if-stack.

      • isNotInInactiveIfBlock

        protected boolean isNotInInactiveIfBlock​(AbstractCodeLine curLine)
        Tests if you are allowed to execute code, based on the current if-block situation. Delivers false, if we are in an if-block whose condition was evaluated as false.
        Returns:
        is code execution allowed?

      • processMarkerCodeLine

        protected boolean processMarkerCodeLine​(AbstractCodeLine line)
                                 throws ImpExException
        Processes the special logic of the marker set to the given code line and returns true, if the code line should be executed or the marker prohibits the execution.
        Parameters:
        line - the code line with set marker
        Returns:
        true, if line has to be executed
        Throws:
        ImpExException - the marker set to the code line is unknown

      • processCodeLine

        protected void processCodeLine​(AbstractCodeLine line)
                        throws ImpExException
        Processes given code line. Checks for special marker logic and executed the code line.
        Parameters:
        line - line to execute
        Throws:
        ImpExException - error while execution of code line

      • setBeanShellContext

        protected void setBeanShellContext​(bsh.Interpreter shell,
                                   java.util.Map<java.lang.Integer,​java.lang.String> line)
                            throws bsh.EvalError
        Declares default variables within the bean shell context which therefore are available within bean shell code interpretation. These variables are:
        • line - the given line object
        • currentLineNumber - the current line number
        • currentLocation - the current location of the active reader
        Parameters:
        shell - bean shell interpreter where the variables will be set
        line - line object which will be set to variable line
        Throws:
        bsh.EvalError - error while setting variables

      • getScriptExecutionContext

        protected java.util.Map<java.lang.String,​java.lang.Object> getScriptExecutionContext​(java.util.Map<java.lang.Integer,​java.lang.String> line)

      • addToBeanShellContext

        protected void addToBeanShellContext​(java.lang.String key,
                                     java.lang.Object value)
                              throws bsh.EvalError
        Adds a variable referencing to the given object to the bean shell context.
        Parameters:
        key - name of variable
        value - object reference
        Throws:
        bsh.EvalError - error with bean shell

      • getFromBeanShellContext

        public java.lang.Object getFromBeanShellContext​(java.lang.String key)
                                         throws bsh.EvalError
        Gets the value of the given variable within bean shell context.
        Parameters:
        key - name of variable
        Returns:
        value of given variable name
        Throws:
        bsh.EvalError - error with bean shell

      • execute

        protected void execute​(AbstractCodeLine line,
                       java.util.Map csvLine)
                throws ImpExException
        Executes the code line at the bean shell.
        Parameters:
        line - the code line
        csvLine - current line object for context variable 'line'
        Throws:
        ImpExException - error while code execution

      • execute

        protected void execute​(AbstractCodeLine line,
                       java.util.Map csvLine,
                       boolean forEachMode)
                throws ImpExException
        Executes the code line at the bean shell.
        Parameters:
        line - the code line
        csvLine - current line object for context variable 'line'
        forEachMode - is the reason for execution not the normal script processing, instead it is a execution reasoned by a forEach marker?
        Throws:
        ImpExException - error while code execution

      • assureBeanShellLoaded

        protected void assureBeanShellLoaded()
                              throws ImpExException
        Assures that the bean shell instance is loaded.
        Throws:
        ImpExException - error while loading shell

      • isDebugEnabled

        public boolean isDebugEnabled()
        Returns:
        is debug mode enabled?

      • debug

        public void debug​(java.lang.String msg,
                  java.lang.Throwable throwable)
        Debugs a message with the current logger instance.
        Parameters:
        msg - debug message
        throwable - cause of message

      • debug

        public void debug​(java.lang.String msg)
        Debugs a message with the current logger instance.
        Parameters:
        msg - debug message

      • isInfoEnabled

        public boolean isInfoEnabled()
        Returns:
        is info mode enabled?

      • info

        public void info​(java.lang.String msg)
        Prints an info message with the current logger instance.
        Parameters:
        msg - info message

      • info

        public void info​(java.lang.String msg,
                 java.lang.Throwable throwable)
        Prints an info message with the current logger instance.
        Parameters:
        msg - info message
        throwable - cause of message

      • warn

        public void warn​(java.lang.String msg)
        Prints an warn message with the current logger instance.
        Parameters:
        msg - warn message

      • warn

        public void warn​(java.lang.String msg,
                 java.lang.Throwable throwable)
        Prints an warn message with the current logger instance.
        Parameters:
        msg - warn message
        throwable - cause of message

      • error

        public void error​(java.lang.String msg)
        Prints an error message with the current logger instance.
        Parameters:
        msg - error message

      • error

        public void error​(java.lang.String msg,
                  java.lang.Throwable throwable)
        Prints an error message with the current logger instance.
        Parameters:
        msg - error message
        throwable - cause of message