Package de.hybris.platform.impex.jalo

Class ImpExReader

java.lang.Object
de.hybris.platform.impex.jalo.ImpExReader
Direct Known Subclasses:
BatchingImportReader, ImpExImportReader, ImportBatchParser.BatchingImpexReader, ScriptValidationReader
public class ImpExReader extends Object
Base class for reading hybris style csv data. This class is able to parse header and value lines. Comments are ignored and variable definitions are inlined. Bean shell commands will be executed if set. Including of external data (from Media, File, InputStream, CSVReader or from a database) is possible via bean shell commands (java statements starting with #%). Attention! This reader only parses value lines. For interpreting them you need to use an extension like ImpExImportReader.
The recommended way of using this reader is:
  1. Instantiate a new reader with a constructor giving a source from where the main script is available. This main script should only contain settings via bean shell commands, header definitions and external data inclusions via bean shell commands, no value lines. For available bean shell commands and usage of them see ImpEx Documentation.

    For example:
    ImpExReader reader = new ImpExReader( new CSVReader( "import.impex" ) );

    Example file: "import.impex"
    "#% impex.setRelaxedMode(false);"
    "INSERT_UPDATE Language";"isocode[unique=true]";"name[lang=en]"
    "#% impex.includeExternalData( new CSVReader(""Language.csv"",""windows-1252""));"

    Example file: "Language.csv"
    ;"de";"German"

  2. Configure the reader for your needs, for example:

    reader.enableCodeExecution( true );
    reader.enableExternalImpExSyntaxParsing( true );

  3. Do subsequent calls to readLine() for parsing and getting the headers and value lines.

    Object nextOutput = null;
    do
    {
    nextOutput = reader.readLine();
    if( nextOutput instanceof HeaderDescriptor )
    {
    //do something with header
    }
    if( nextOutput instanceof ValueLine )
    {
    //do something with line
    }
    }
    while(nextOutput != null);

  4. Close the reader.

    reader.close();

  • Field Details

    • FIRST

      public static final Integer FIRST
      The position of current header mode and type within the current line Map.

  • Constructor Details

    • ImpExReader

      public ImpExReader(String fileName, String encoding, boolean skipValueLines) throws UnsupportedEncodingException, FileNotFoundException
      Instantiates a new reader using the file with given name and encoding as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
      Parameters:
      fileName - the name of the file used as input for the reader
      encoding - used encoding for reading the file
      skipValueLines - if true, all value lines (lines within input containing data) will be not interpreted and therefore not returned by call of readLine()
      Throws:
      UnsupportedEncodingException - the given encoding is unknown
      FileNotFoundException - there is no file with the given file name

    • ImpExReader

      public ImpExReader(InputStream input, String encoding) throws UnsupportedEncodingException
      Instantiates a new reader using the given stream and encoding as input. Skipping of value lines will be not enabled, relaxed mode is disabled and a new document ID registry will be instantiated.
      Parameters:
      input - stream from which the script is read
      encoding - used encoding for reading from stream
      Throws:
      UnsupportedEncodingException - encoding is unknown

    • ImpExReader

      public ImpExReader(InputStream input, String encoding, boolean skipValueLines) throws UnsupportedEncodingException
      Instantiates a new reader using the given stream and encoding as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
      Parameters:
      input - stream from which the script is read
      encoding - used encoding for reading from stream
      skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
      Throws:
      UnsupportedEncodingException - encoding is unknown

    • ImpExReader

      public ImpExReader(CSVReader reader, boolean skipValueLines)
      Instantiates a new reader using the given CSVReader as input. Relaxed mode is disabled and a new document ID registry will be instantiated.
      Parameters:
      reader - the reader from which the script is read
      skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them

    • ImpExReader

      public ImpExReader(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode)
      Instantiates a new reader using the given CSVReader as input. A new document ID registry will be instantiated.
      Parameters:
      reader - the reader from which the script is read
      skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
      headerValidationMode - the following modes are supported:
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT

    • ImpExReader

      public ImpExReader(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode, DocumentIDRegistry docIdRegistry)
      Instantiates a new reader using the given CSVReader as input.
      Parameters:
      reader - the reader from which the script is read
      skipValueLines - if true, readLine will not return a ValueLine, they will be skipped without parsing them
      headerValidationMode - the following modes are supported:
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT
      • docIdRegistry - the used document ID registry set to each created ImpExHeaderDesriptor

    • ImpExReader

      public ImpExReader(CSVReader reader, boolean skipValueLines, EnumerationValue headerValidationMode, DocumentIDRegistry docIdRegistry, InvalidHeaderPolicy headerHandlingPolicy)

  • Method Details

    • getInvalidHeaderPolicy

      public InvalidHeaderPolicy getInvalidHeaderPolicy()

    • setInvalidHeaderPolicy

      protected void setInvalidHeaderPolicy(InvalidHeaderPolicy invalidHeaderPolicy)

    • getStrictMode

      protected EnumerationValue getStrictMode()

    • getDefaultReplacements

      protected List<Object[]> getDefaultReplacements()

    • sortRulesList

      protected void sortRulesList(List<Object[]> rulesList)

    • addHeaderReplacementRule

      public void addHeaderReplacementRule(String srcPattern, String tgtPattern, int prio)

    • setAttributeConstraintFilter

      public void setAttributeConstraintFilter(HeaderDescriptor.AttributeConstraintFilter filter)
      Sets a new constraint filter (for defining methods like isMandatory for attributes) used in each created HeaderDescriptor from now.
      Parameters:
      filter - the new filter used in headers

    • getAttributeConstraintFilter

      public HeaderDescriptor.AttributeConstraintFilter getAttributeConstraintFilter()
      Get the current constraint filter used in HeaderDescriptors created until yet.
      Returns:
      the current constraint filter

    • parseHeader

      public static HeaderDescriptor parseHeader(String headerLine) throws ImpExException
      Generates a header object from given line. Relaxed mode is set to false and a new document ID registry is created.
      Parameters:
      headerLine - the line which will be parsed and a header object is created from
      Returns:
      a header object representing the given line
      Throws:
      ImpExException - the given line does not represent a header or is misspelled

    • parseHeader

      public static HeaderDescriptor parseHeader(String headerLine, DocumentIDRegistry docIdRegistry) throws ImpExException
      Generates a header object from given line. Relaxed mode is set to false.
      Parameters:
      headerLine - the line which will be parsed and a header object is created from
      docIdRegistry - a document ID registry which will be attached to the header object
      Returns:
      a header object representing the given line
      Throws:
      ImpExException - the given line does not represent a header or is misspelled

    • parseHeader

      public static HeaderDescriptor parseHeader(String headerLine, EnumerationValue mode) throws ImpExException
      Generates a header object from given line. A new document ID registry is created.
      Parameters:
      headerLine - the line which will be parsed and a header object is created from
      mode - should the relaxed mode be set in header object
      Returns:
      a header object representing the given line
      Throws:
      ImpExException - the given line does not represent a header or is misspelled

    • parseHeader

      public static HeaderDescriptor parseHeader(String headerLine, EnumerationValue mode, DocumentIDRegistry docIdRegistry) throws ImpExException
      Generates a header object from given line.
      Parameters:
      headerLine - the line which will be parsed and a header object is created from
      mode - should the relaxed mode be set in header object
      docIdRegistry - a document ID registry which will be attached to the header object
      Returns:
      a header object representing the given line
      Throws:
      ImpExException - the given line does not represent a header or is misspelled

    • isExternalSyntaxParsingEnabled

      public boolean isExternalSyntaxParsingEnabled()
      Will non value lines in external lines be interpreted or parsed as value lines.
      Returns:
      is external syntax parsing activated?

    • enableExternalImpExSyntaxParsing

      @Deprecated(since="ages", forRemoval=false) public void enableExternalImpExSyntaxParsing(boolean isOn)
      Deprecated.
      since ages - useenableExternalSyntaxParsing(boolean) instead
      Sets the external syntax parsing flag. If is not set, syntax in external data like headers can not be interpreted.
      Parameters:
      isOn - set true if external parsing should be activated

    • enableExternalSyntaxParsing

      public void enableExternalSyntaxParsing(boolean isOn)
      Sets the external syntax parsing flag. If is not set, syntax in external data like headers can not be interpreted.
      Parameters:
      isOn - set true if external parsing should be activated

    • isCombinedSearchEnabled

      public boolean isCombinedSearchEnabled()
      Tells whether or not to use combined search queries. This means that all item reference expressions of a single line are queried using *one* big UNION query instead of on query per single reference expression. This avoids database round trips and possibly safes cpu in database itself.
      Returns:
      true if config parameter

    • setLocale

      public void setLocale(Locale locale)
      Sets the locale used while parsing value lines. Locale is important in case of numbers and dates.
      Parameters:
      locale - the new locale used from now

    • getLocale

      public Locale getLocale()
      Get the current locale setting used for parsing value lines.
      Returns:
      current locale setting

    • close

      public void close() throws IOException
      Closes all used readers.
      Throws:
      IOException - error while closing streams

    • getCSVReader

      public CSVReader getCSVReader()
      Gets the reader instance of the main script. To get the current reader call getCurrentReader().
      Returns:
      main reader

    • addExternalDataMedia

      public void addExternalDataMedia(ImpExMedia ext)
      Adds an external media to the reader context which is then accessible via includeExternalDataMedia.
      Parameters:
      ext - the new media which can be included as external data source now

    • addExternalDataMedias

      public void addExternalDataMedias(Collection<ImpExMedia> toAdd)
      Adds an collection of external medias to the reader context which are then accessible via includeExternalDataMedia each.
      Parameters:
      toAdd - the new medias which can be included as external data source now

    • removeExternalDataMedia

      public void removeExternalDataMedia(Media ext)
      Removes an media from the reader context and is therefore not includable as external data anymore.
      Parameters:
      ext - the media which will be removed from context

    • removeExternalDataMedias

      public void removeExternalDataMedias(Collection<Media> toRemove)
      Removes an collection medias from the reader context and are therefore not integrable as external data anymore.
      Parameters:
      toRemove - the media collection which will be removed from context

    • getAllExternalDataMediaCodes

      protected Set<String> getAllExternalDataMediaCodes()
      Gets all medias added to reader context for external data inclusion.
      Returns:
      set of all integrable medias

    • findExternalDataMedia

      public ImpExMedia findExternalDataMedia(String code) throws JaloBusinessException
      Searches the media with the given code in the context of the reader and returns it.
      Parameters:
      code - code of the media
      Returns:
      media with given code or null if no data is found
      Throws:
      JaloBusinessException - a media with the code was found but has had no data attached

    • findExternalDataMedia

      protected ImpExMedia findExternalDataMedia(String prefix, String code)

    • isIncludingExternalData

      public boolean isIncludingExternalData()
      Tells whether or not this reader fetches its data from a external source currently (see includeExternalData(CSVReader) etc. ).
      Returns:
      are we are reading external data at the moment

    • setCurrentHeader

      public void setCurrentHeader(String headerCSVDefinition) throws ImpExException
      Generates from given header line a HeaderDescriptor and sets it as current header for further reading of input.
      Parameters:
      headerCSVDefinition - line defining a header
      Throws:
      ImpExException - error while generation of the header object

    • setCurrentHeader

      public void setCurrentHeader(HeaderDescriptor header)
      Sets the given header instance as current header. The current header is set for each read value line.
      Parameters:
      header - the header which will be used from now as header for read value lines

    • getCurrentHeader

      public HeaderDescriptor getCurrentHeader()
      Gets the current header which was set at each read value line.
      Returns:
      current header instance, or null if no header is activated (at start of reading)

    • initDatabase

      public final void initDatabase(String url, String user, String password, String className)
      Initializes the parameters for using a database connection using includeSQLData.
      Parameters:
      url - url to the database
      user - name of user / account in database
      password - password of user
      className - name of database driver (JDBC)

    • includeSQLData

      public void includeSQLData(String url, String user, String password, String className, String sqlStatement)
      Establishes a connection to a database with the given parameters, fires the given statement and includes the resulting result set as external data. Each line of the result set will be parsed as value line.

      Example script:
      #% impex.enableCodeExecution( true );
      #% impex.setRelaxedMode( true );
      INSERT_UPDATE Product; code[unique=true]
      #% impex.includeSQLData("jdbc:mysql://localhost/testdb?user=hendrik&password=hendrik" ,"com.mysql.jdbc.Driver","Select * from Products");

      Parameters:
      url - url to the database
      user - name of the user / account at database
      password - password of user
      className - name of the database driver
      sqlStatement - error while establishing connection or reading result set

    • includeSQLData

      public void includeSQLData(String sqlStatement)
      Uses current settings to establish a database connection, to fire given statement and to parse the result set to value lines.
      Parameters:
      sqlStatement - error while accessing the result set of given statement

    • includeSQLData

      public void includeSQLData(String sqlStatement, int skip, int offset)
      Fires the given statement to the database connection established with current settings and parses the lines of the resulting result set as value lines.

      Example script:
      #% impex.enableCodeExecution( true );
      #% impex.setRelaxedMode( true );
      #% impex.initDatabase( "jdbc:mysql://localhost/testdb?user=hendrik&password=hendrik","com.mysql.jdbc.Driver" );
      INSERT_UPDATE Product; code[unique=true]
      #% impex.includeSQLData("Select * from Products");

      Parameters:
      sqlStatement - statement which will be executed
      skip - lines to skip within result set
      offset - amount of columns to skip from left

    • includeExternalData

      public void includeExternalData(File file, String encoding, int linesToSkip) throws UnsupportedEncodingException, FileNotFoundException
      Creates a new temporary reader with given file and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. Reading from temporary reader can be stopped by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
      Parameters:
      file - file which contains external data to be included
      encoding - the encoding used in given file
      linesToSkip - amount of lines the reader of the file will initial skip
      Throws:
      UnsupportedEncodingException - given encoding is unknown
      FileNotFoundException - given file is not found

    • includeExternalData

      public void includeExternalData(InputStream inputStream, String encoding, int linesToSkip) throws UnsupportedEncodingException
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
      Parameters:
      inputStream - input stream from which external data to be included is accessable
      encoding - the used encoding
      linesToSkip - amount of lines the reader of the stream will initial skip
      Throws:
      UnsupportedEncodingException - given encoding is unknown

    • includeExternalData

      public void includeExternalData(InputStream inputStream, String encoding, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow.
      Parameters:
      inputStream - input stream from which external data to be included is accessible
      encoding - the used encoding
      linesToSkip - amount of lines the reader of the stream will initial skip
      columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because the data columns do not have a first column with type information. So the header columns have to be shifted one to the left =>-1.
      Throws:
      UnsupportedEncodingException - given encoding is unknown

    • includeExternalData

      public void includeExternalData(InputStream inputStream, String encoding, char[] delimiter, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow.
      Parameters:
      inputStream - input stream from which external data to be included is accessible
      encoding - the used encoding
      delimiter - field separators used in external data
      linesToSkip - amount of lines the reader of the stream will initial skip
      columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because the data columns do not have a first column with type information. So the header columns have to be shifted one to the left =>-1.
      Throws:
      UnsupportedEncodingException - given encoding is unknown

    • includeExternalData

      public void includeExternalData(String fileName, String fileEncoding, int linesToSkip) throws UnsupportedEncodingException, FileNotFoundException
      Creates a new temporary reader with file of given filename and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. Reading from temporary reader will stop by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
      Parameters:
      fileName - file which contains external data to be included
      fileEncoding - the encoding used in given file
      linesToSkip - amount of lines the reader of the file will initial skip
      Throws:
      UnsupportedEncodingException - given encoding is unknown
      FileNotFoundException - given file is not found

    • includeExternalData

      public void includeExternalData(CSVReader reader)
      Redirects the main reader to the given temporary reader for including external data. So you can include external data given with the reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically). The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
      Parameters:
      reader - the temporary reader which reads from external data

    • includeExternalData

      public void includeExternalData(CSVReader reader, int linesToSkip, int columnOffset)
      Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
      Parameters:
      reader - the temporary reader which reads from external data
      linesToSkip - amount of lines which will be skipped when start reading from external data
      columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.

    • includeExternalData

      public void includeExternalData(CSVReader reader, int columnOffset)
      Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
      Parameters:
      reader - the temporary reader which reads from external data
      columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.

    • includeExternalData

      public void includeExternalData(CSVReader reader, int columnOffset, String locationText)
      Redirects the main reader to the given temporary reader for including external data. So you can include external data given with a reader within the main reader flow. Reading from given temporary reader will stop by reaching the end of stream (switches back automatically).
      Parameters:
      reader - the temporary reader which reads from external data
      columnOffset - position difference of header columns to data columns. Is data in hybris style, use 0 because data columns positions are exactly the same as in header. In default style use -1, because then data columns do not have a first column with type information as in hybris style. So the header columns have to be shifted one to the left => -1.
      locationText - identifier of the external data for logging purposes

    • includeExternalData

      public void includeExternalData(InputStream inputStream, String encoding, boolean setNewHeader) throws UnsupportedEncodingException, ImpExException
      Creates a new temporary reader with given input stream and encoding and redirects the main reader to that temporary reader for including external data. So you can include external data within the main reader flow. The column offset will be set to -1, so normal CSV data is assumed (not in hybris style).
      Parameters:
      inputStream - input stream from which external data to be included is accessible
      encoding - the used encoding
      setNewHeader - if true, the first line of the external data will be tried to interpreted as header
      Throws:
      UnsupportedEncodingException - given encoding is unknown
      ImpExException - error while parsing header from first line (only when setNewHeader==true)

    • includeExternalDataMedia

      public void includeExternalDataMedia(String code) throws UnsupportedEncodingException
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped and column offset is set to -1 (data is not in hybris style).
      Parameters:
      code - the media code
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      public void includeExternalDataMedia(String code, int columnOffset) throws UnsupportedEncodingException
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped.
      Parameters:
      code - the media code
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      @Deprecated(since="ages", forRemoval=false) public void includeExternalDataMedia(String code, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
      The given linesToSkip value will be used instead of the equivalent attribute set at the media. It is preferred to set the encoding and linesToSkip directly at the media and to use includeExternalDataMedia(String, int) instead.
      Parameters:
      code - the media code
      linesToSkip - amount of lines which will be skipped when start reading from external data
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      @Deprecated(since="ages", forRemoval=false) public void includeExternalDataMedia(String code, String encoding) throws UnsupportedEncodingException
      Deprecated.
      since ages - useincludeExternalDataMedia(String) instead
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped and column offset is set to -1 (data is not in hybris style).
      The given encoding will be used instead of the encoding set at the media. It is preferred to set the encoding directly at the media and to use includeExternalDataMedia(String) instead.
      Parameters:
      code - the media code
      encoding - the encoding of the csv data stored inside the media
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      @Deprecated(since="ages", forRemoval=false) public void includeExternalDataMedia(String code, String encoding, int columnOffset) throws UnsupportedEncodingException
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before. No lines will be skipped.
      The given encoding will be used instead of the encoding set at the media. It is preferred to set the encoding directly at the media and to use includeExternalDataMedia(String) instead.
      Parameters:
      code - the media code
      encoding - the encoding of the csv data stored inside the media
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      @Deprecated(since="ages", forRemoval=false) public void includeExternalDataMedia(String code, String encoding, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
      The given encoding and linesToSkip value will be used instead of the equivalent attribute set at the media. It is preferred to set the encoding and linesToSkip directly at the media and to use includeExternalDataMedia(String) instead.
      Parameters:
      code - the media code
      encoding - the encoding of the csv data stored inside the media
      linesToSkip - amount of lines which will be skipped when start reading from external data
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      @Deprecated(since="ages", forRemoval=false) public void includeExternalDataMedia(String code, String encoding, char delimiter, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Deprecated.
      since ages - useincludeExternalDataMedia(String, int) instead
      Redirects the import reader to external data stored inside a media which was announced with addExternalDataMedia before.
      The given encoding,delimiter and linesToSkip values will be used instead of the equivalent attribute set at the media. It is preferred to set these values directly at the media and to use includeExternalDataMedia(String) instead.
      Parameters:
      code - the media code
      encoding - the encoding of the csv data stored inside the media
      delimiter - field separator used in external data
      linesToSkip - amount of lines which will be skipped when start reading from external data
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      public void includeExternalDataMedia(ImpExMedia media) throws UnsupportedEncodingException
      Redirects the import reader to external data stored inside a media. ColumnOffset will be set to -1
      Parameters:
      media - the media containing the external data
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      public void includeExternalDataMedia(ImpExMedia media, int columnOffset) throws UnsupportedEncodingException
      Redirects the import reader to external data stored inside a media.
      Parameters:
      media - the media containing the external data
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • includeExternalDataMedia

      public void includeExternalDataMedia(ImpExMedia media, String encoding, char delimiter, int linesToSkip, int columnOffset) throws UnsupportedEncodingException
      Redirects the import reader to external data stored inside a media.
      ATTENTION: The given parameters will be used for configuring of media, not the attributes set at media. So if you have set the encoding attribute at the media it will be ignored and the given encoding will be used instead.
      Parameters:
      media - media instance holding the external data
      encoding - the encoding of the csv data stored inside the media
      delimiter - field separator used in external data
      linesToSkip - amount of lines which will be skipped when start reading from external data
      columnOffset - the column offset compared to impex standard: if the first column already contains data use -1 since impex normally starts in column 0
      Throws:
      UnsupportedEncodingException - unknown encoding given

    • getCurrentLocation

      public String getCurrentLocation()
      Gets the current line number of the reader. If the current reader is temporary (reads external data) the location in external data is appended too.
      Returns:
      current line number of reader

    • getCurrentReader

      protected CSVReader getCurrentReader()
      Returns the current activated reader used for reading data. It will return the main reader or the current temporary reader if external data is read.
      Returns:
      currently active reader

    • getDocumentIDRegistry

      public DocumentIDRegistry getDocumentIDRegistry()
      Gets the used document ID registry which will be set to each created header.
      Returns:
      current registry

    • gotInsertedLines

      protected boolean gotInsertedLines()
      Are lines available which have to be used before reading from reader?
      Returns:
      true if inserted lines are available

    • insertLine

      public void insertLine(Map csvLine)
      Inserts the line into reading process. The line will be processed before reading next line from reader.
      Parameters:
      csvLine - line which will be inserted in line processing

    • readNextCSVLine

      protected Map<Integer,String> readNextCSVLine()
      Reads the next line. It will be read from buffer with inserted lines or when empty from active reader. The line is a map consisting of each read unescaped column field (determined by field separator set at the used reader) mapped to the column position in associated header.
      Returns:
      next read line

    • adjustLineIndexes

      protected Map<Integer,String> adjustLineIndexes(Map<Integer,String> line)
      Adjusts the line indexes using the column offset set at reader manager. The key values will be summed with the set column offset.
      Parameters:
      line - line to adjust
      Returns:
      adjusted line

    • readLine

      public Object readLine() throws ImpExException
      Fetches the next line, parses it and returns it encapsulated in a HeaderDescriptor if the line was a header definition or a ValueLine if it was a data row.
      Returns:
      either HeaderDescriptor or ValueLine instances or null if end of data is reached.
      Throws:
      ImpExException - if a header could not be parsed properly

    • addHeaderExceptionInfoAsComment

      protected void addHeaderExceptionInfoAsComment(Map<Integer,String> adjustedLine, HeaderValidationException ex)

    • createInvalidHeader

      public HeaderDescriptor createInvalidHeader(Map<Integer,String> line, HeaderValidationException ex) throws HeaderValidationException
      Throws:
      HeaderValidationException

    • hasCellDecorators

      protected boolean hasCellDecorators()
      Checks if there are decorators set for current header.
      Returns:
      true if decorators are set

    • getCellDecorators

      protected Map<Integer,CSVCellDecorator> getCellDecorators()
      Gets all decorators set for current header.
      Returns:
      decorators set to current header

    • setCellDecorators

      protected void setCellDecorators(HeaderDescriptor header)
      Extracts all decorators from given header columns and stores them for line parsing.
      Parameters:
      header - header from which the decorators will be extracted

    • storeUserRightsLine

      protected void storeUserRightsLine(Map line)
      Concatenates given line and adds it to the already read user rights lines for further special processing.
      Parameters:
      line - line with a UserRight data row

    • writeUserRightsLines

      protected void writeUserRightsLines() throws ImpExException
      Writes all stored UserRights lines to an temporary buffer and imports this file with the ImportExportUserRightsHelper.
      Throws:
      ImpExException - there were no user rights for import.

    • addDefinition

      protected void addDefinition(String cell)
      Adds a definition to the reader context and is therefore available for parsing of headers. Is a definition with same key already present it will be replaced by this one.
      Parameters:
      cell - string containing the definition

    • splitDefinitonCell

      protected String[] splitDefinitonCell(String cell)
      Splits the given definition in key and value pair.
      Parameters:
      cell - the definition to split
      Returns:
      array consisting of two fields key and value: String[{key},{value}]

    • checkDefinitonKey

      protected void checkDefinitonKey(String newKey, Map currentDefinitions) throws ImpExException
      Checks if a key of a definition is a prefix of an already present key. If no clash exist, the method will pass, else throws an exception.
      Parameters:
      newKey - key to check
      currentDefinitions - map with present definition
      Throws:
      ImpExException - the given key is a prefix of a present key

    • replaceDefinitions

      protected Map<Integer,String> replaceDefinitions(Map<Integer,String> line)
      Checks the given line for definition usage and replace all definition keys with the correct definition values.
      Parameters:
      line - line to check
      Returns:
      line with replaced definition keys

    • findMarker

      protected String findMarker(String expr)
      Checks if the given expression starts with a marker (special construct, see ImpExConstants.Syntax.CodeMarkers ) and returns the marker string..
      Parameters:
      expr - the expression to check
      Returns:
      the marker string if the expression starts with it or null

    • createCodeLine

      protected AbstractCodeLine createCodeLine(Map<Integer,String> line)
      Interprets the given line as code line. Therefore, it cuts code declaration (#%) in first column, cuts and remembers defined markers and constructs a code line object. It will return a code line object always, even there is no code declaration, so you have to check first using isCodeLine(Map).
      Parameters:
      line - line to interpret
      Returns:
      code line instance representing this line

    • createCodeLineLegacyWay

      protected AbstractCodeLine createCodeLineLegacyWay(Map<Integer,String> line)

    • createCodeLineModernWay

      protected AbstractCodeLine createCodeLineModernWay(Map<Integer,String> line)

    • createNewHeader

      protected HeaderDescriptor createNewHeader(Map<Integer,String> line) throws HeaderValidationException
      Creates from a header line a header object. It is not checked, if the line describes a header.
      Parameters:
      line - line for which a header object is needed
      Returns:
      created header object
      Throws:
      HeaderValidationException - header is not valid

    • applyHeaderReplacements

      protected List<String> applyHeaderReplacements(List<String> columns)

    • getLineSize

      protected int getLineSize(Map<Integer,String> line)
      Returns the actual length of a value line. Since a line contains map entries only not all cell must be filled so line.size() is not sufficient for that!
      Parameters:
      line - line to check
      Returns:
      index of last column in line (column with largest index)

    • lineToList

      protected List<String> lineToList(Map<Integer,String> line)
      Rearranges a csv line map into a list object. Cell values are copied into a new list where each column with index x will be placed in list bucket x.
      Parameters:
      line - line to transform
      Returns:
      list of columns with null entries for non defined indexes

    • isEmptyLine

      protected boolean isEmptyLine(Map<Integer,String> line)
      Checks if line has no columns.
      Parameters:
      line - line to check
      Returns:
      true, if line has no columns

    • setValidationMode

      public void setValidationMode(String mode)

    • setValidationMode

      public void setValidationMode(EnumerationValue headerValidationMode)
      Sets the header valdiation mode used for all created headers.
      Parameters:
      headerValidationMode - the following modes are supported:
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_STRICT
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.IMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_ONLY
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_RELAXED
    • ImpExConstants.Enumerations.ImpExValidationModeEnum.EXPORT_REIMPORT_STRICT

    • getValidationMode

      public EnumerationValue getValidationMode()

    • setRelaxedMode

      @Deprecated(since="3.0", forRemoval=false) public void setRelaxedMode(boolean enable)
      Deprecated.
      since 3.0, use setValidationMode(EnumerationValue) instead

    • setRelaxedMode

      @Deprecated(since="3.0", forRemoval=false) public void setRelaxedMode(String enable)
      Deprecated.
      since 3.0, use setValidationMode(EnumerationValue) instead

    • isCommentLine

      protected boolean isCommentLine(Map<Integer,String> line)
      Checks if the given line is a comment (introduced with #).
      Parameters:
      line - line to check
      Returns:
      true, if line is a comment (starts with #)

    • isCodeLine

      protected boolean isCodeLine(Map<Integer,String> line)
      Checks if the given line is a code line (introduced with #%).
      Parameters:
      line - line to check
      Returns:
      true, if line is a code line (starts with #%)

    • isHeaderLine

      protected boolean isHeaderLine(Map<Integer,String> line)
      Checks if the given line is a header line (introduced with header mode string).
      Parameters:
      line - line to check
      Returns:
      true, if line is a header line

    • isDefinition

      protected boolean isDefinition(Map<Integer,String> line)
      Checks if the given line declares a definition.
      Parameters:
      line - line to check
      Returns:
      true, if line declares a definition

    • isStartUserRights

      protected boolean isStartUserRights(Map<Integer,String> line)
      Checks if the given introduces a UserRight section (consists only of $START_USERRIGHTS).
      Parameters:
      line - line to check
      Returns:
      true, if line introduces a UserRight section

    • isEndUserRights

      protected boolean isEndUserRights(Map<Integer,String> line)
      Checks if the given ends a UserRight section (consists only of $END_USERRIGHTS).
      Parameters:
      line - line to check
      Returns:
      true, if line ends a UserRight section

    • createValueLine

      protected ValueLine createValueLine(HeaderDescriptor header, Map<Integer,String> line)
      Creates from a line which represents a data row a value line.
      Parameters:
      header - the header which corresponds to the given line
      line - line for which a value line is needed
      Returns:
      value line created from given line

    • isCodeExecutionEnabled

      public final boolean isCodeExecutionEnabled()
      Is parsing and interpreting of code lines in data enabled?
      Returns:
      code parsing is enabled

    • enableCodeExecution

      public void enableCodeExecution(boolean isOn)
      Sets the code execution flag. Is code execution enabled, the parsing and interpreting of code lines is enabled. Otherwise these lines will be seen as comments.
      Parameters:
      isOn - Should code execution be enabled?

    • isExternalCodeExecutionEnabled

      public final boolean isExternalCodeExecutionEnabled()
      Is parsing and interpreting of code lines in external data enabled?
      Returns:
      code parsing and interpreting is enabled

    • enableExternalDataCodeExecution

      @Deprecated(since="ages", forRemoval=false) public void enableExternalDataCodeExecution(boolean isOn)
      Deprecated.
      since ages - useenableExternalCodeExecution(boolean) instead
      Sets the external code execution flag. Is external code execution enabled, the parsing and interpreting of code lines in external data is enabled. Otherwise these lines will be seen as comments.
      Parameters:
      isOn - Should external code execution be enabled?

    • enableExternalCodeExecution

      public void enableExternalCodeExecution(boolean isOn)
      Sets the external code execution flag. Is external code execution enabled, the parsing and interpreting of code lines in external data is enabled. Otherwise these lines will be seen as comments.
      Parameters:
      isOn - Should external code execution be enabled?

    • setBeforeEachCode

      protected void setBeforeEachCode(AbstractCodeLine line)
      Sets the code line executed after parsing but before interpreting of each line.
      Parameters:
      line - code line to remember for before execution

    • getBeforeEachCode

      public AbstractCodeLine getBeforeEachCode()
      Gets the code line executed after parsing but before interpreting of each line.
      Returns:
      code line of beforeEach marker

    • enterIfBlock

      public boolean enterIfBlock(AbstractCodeLine line)
      Has to be called when an if-condition occurs. Puts the current code line onto the if-stack. If there is already an 'if' on the stack and is this if is inactive (related if-condition was evaluated false) then the result was the current if-condition is also set to false ignoring the real result, because an if-block within an inactive if-block can must not be executed.
      Parameters:
      line - the current code line containing the if-condition

    • exitIfBlock

      public void exitIfBlock()
      Has to be called when an endif-marker occurs. Pops the current if code line from the if-stack.

    • isNotInInactiveIfBlock

      protected boolean isNotInInactiveIfBlock(AbstractCodeLine curLine)
      Tests if you are allowed to execute code, based on the current if-block situation. Delivers false, if we are in an if-block whose condition was evaluated as false.
      Returns:
      is code execution allowed?

    • processMarkerCodeLine

      protected boolean processMarkerCodeLine(AbstractCodeLine line) throws ImpExException
      Processes the special logic of the marker set to the given code line and returns true, if the code line should be executed or the marker prohibits the execution.
      Parameters:
      line - the code line with set marker
      Returns:
      true, if line has to be executed
      Throws:
      ImpExException - the marker set to the code line is unknown

    • processCodeLine

      protected void processCodeLine(AbstractCodeLine line) throws ImpExException
      Processes given code line. Checks for special marker logic and executed the code line.
      Parameters:
      line - line to execute
      Throws:
      ImpExException - error while execution of code line

    • setBeanShellContext

      protected void setBeanShellContext(bsh.Interpreter shell, Map<Integer,String> line) throws bsh.EvalError
      Declares default variables within the bean shell context which therefore are available within bean shell code interpretation. These variables are:
      • line - the given line object
      • currentLineNumber - the current line number
      • currentLocation - the current location of the active reader
      Parameters:
      shell - bean shell interpreter where the variables will be set
      line - line object which will be set to variable line
      Throws:
      bsh.EvalError - error while setting variables

    • getScriptExecutionContext

      protected Map<String,Object> getScriptExecutionContext(Map<Integer,String> line)

    • addToBeanShellContext

      protected void addToBeanShellContext(String key, Object value) throws bsh.EvalError
      Adds a variable referencing to the given object to the bean shell context.
      Parameters:
      key - name of variable
      value - object reference
      Throws:
      bsh.EvalError - error with bean shell

    • getFromBeanShellContext

      public Object getFromBeanShellContext(String key) throws bsh.EvalError
      Gets the value of the given variable within bean shell context.
      Parameters:
      key - name of variable
      Returns:
      value of given variable name
      Throws:
      bsh.EvalError - error with bean shell

    • execute

      protected void execute(AbstractCodeLine line, Map csvLine) throws ImpExException
      Executes the code line at the bean shell.
      Parameters:
      line - the code line
      csvLine - current line object for context variable 'line'
      Throws:
      ImpExException - error while code execution

    • execute

      protected void execute(AbstractCodeLine line, Map csvLine, boolean forEachMode) throws ImpExException
      Executes the code line at the bean shell.
      Parameters:
      line - the code line
      csvLine - current line object for context variable 'line'
      forEachMode - is the reason for execution not the normal script processing, instead it is a execution reasoned by a forEach marker?
      Throws:
      ImpExException - error while code execution

    • modernExecute

      protected void modernExecute(AbstractScriptingEngineCodeLine line, Map csvLine) throws ImpExException
      Throws:
      ImpExException

    • legacyExecute

      protected void legacyExecute(AbstractCodeLine line, Map<Integer,String> csvLine) throws ImpExException
      Throws:
      ImpExException

    • assureBeanShellLoaded

      protected void assureBeanShellLoaded() throws ImpExException
      Assures that the bean shell instance is loaded.
      Throws:
      ImpExException - error while loading shell

    • isDebugEnabled

      public boolean isDebugEnabled()
      Returns:
      is debug mode enabled?

    • debug

      public void debug(String msg, Throwable throwable)
      Debugs a message with the current logger instance.
      Parameters:
      msg - debug message
      throwable - cause of message

    • debug

      public void debug(String msg)
      Debugs a message with the current logger instance.
      Parameters:
      msg - debug message

    • isInfoEnabled

      public boolean isInfoEnabled()
      Returns:
      is info mode enabled?

    • info

      public void info(String msg)
      Prints an info message with the current logger instance.
      Parameters:
      msg - info message

    • info

      public void info(String msg, Throwable throwable)
      Prints an info message with the current logger instance.
      Parameters:
      msg - info message
      throwable - cause of message

    • warn

      public void warn(String msg)
      Prints an warn message with the current logger instance.
      Parameters:
      msg - warn message

    • warn

      public void warn(String msg, Throwable throwable)
      Prints an warn message with the current logger instance.
      Parameters:
      msg - warn message
      throwable - cause of message

    • error

      public void error(String msg)
      Prints an error message with the current logger instance.
      Parameters:
      msg - error message

    • error

      public void error(String msg, Throwable throwable)
      Prints an error message with the current logger instance.
      Parameters:
      msg - error message
      throwable - cause of message