The operator's documentation should be created in a file named README.md in the operator's folder. For example, to create the documentation for the dummy operator AppendString, you have to create a new README.md file in the following path:

{ROOT}/subengines/com/sap/python27/operators/com/mydomain/util/appendString/README.md

The documentation is written using Markdown syntax (more information here). You can check the result by right-clicking your operator and choosing Open Documentation, in SAP Data Hub Modeler's UI.

You can also add a custom icon for your operator. To do this, first copy the icon to the operator's folder, for example:

{ROOT}/subengines/com/sap/python27/operators/com/mydomain/util/appendString/icon.png

Then, you have to make sure your _get_operator_info method is setting icon.png (or another file) as the iconsrc field. An example is shown below:

def _get_operator_info(self):
        ...
        return OperatorInfo(...,
                            iconsrc="icon.png",
                            ...)

Lastly, you must regenerate the operator's json.