Eclipse project generator

The common problem of all cross-compiled projects is well-configured IDE. Usually, the first thing done at the beginning of the project is cloning the repository and setting up a build environment. If you’re patient enough, probably you can configure include configuration of your favorite code editor. If it’s a simple project this approach may be suitable. But if your code gets more and more complicated, new libraries are attached, your IDE will be full of red underscored code. Usually, it’s treated as one-time work, however, what will happen if your hard drive is gone, or if you want to work on several repositories simultaneously? Simple copying of IDE configuration sometimes works, sometimes not. Especially if you are setting up an environment on a new machine, include path problems may take you many hours of stupid work. Some projects setting is almost impossible by a human.

Some time ago I’ve realized that all you need to set up your environment are build variables passed to the compiler. Usually, all of it is placed in the Makefile script. It’s a common tool for most of the cross-compiled projects – the place where compiler, configuration parameters, library paths, and many other things meet to build your code. This implies that the Makefile script is the best place to set up your IDE. This is the reason for creating this simple Python script. I made some reverse engineering to see how CDT Eclipse configuration is stored in its XML files (.cproject, .project, and .settings directory). The general principle is simple and filling it with automatically generated data was not a big deal.

The first thing, which is not a part of a script, is copying .settings folder with it’s content to a project directory. I didn’t spend time investigating what is it about – it is a good googling topic – but it is an obligatory part of the Eclipse project. I suppose that local project settings, not connected with the build system are set up here. So let’s execute our first step:

$ cp -r template/.settings-base project/.settings

After that, we can start the most interesting part – analyzing what’s important for Eclipse to parse your C/C++ code correctly. Script base principle is loading .cproject and .project bare template into ElementTree structure and filling it with data taken from parameters. The script will be executed in Makefile so all the parameters will be passed to it like in any other program e.g. gcc. I will give short snippets of Python in the article, the full-featured script is added on the bottom.

Loading .project file stub is done with the following commands:

import xml.etree.ElementTree as ET

tree = ET.parse("template/.project-stub")
root = tree.getroot()

At this point, we have loaded and parsed XML structure of standard .project file in a tree object. Root node handle is fetched in the last line of a snippet. .project file contains language-neutral settings like name of a project – this is the first thing filled with our data:

projectNameNode = root.find('name')
projectNameNode.text = args.name[0]

find method gets the node with the specified identifier, which in our case is name. The second line set its content. Worth noting here is the usage of argparse Python module. At the beginning of the script, you will find

import argparse

parser = argparse.ArgumentParser(description='Eclipse project  generator, creates .project and .cproject files')
parser.add_argument('--name', nargs=1, metavar='NAME', help='Project name, will be displayed in project explorer')
parser.add_argument('--includes', nargs='+', metavar='PATH', help='Paths to include directories, might be relative')
parser.add_argument('--sources', nargs='+', metavar='PATH', help='Paths to source locations, relative to project only!')
parser.add_argument('--external', nargs='*', metavar='PATH', help='Source directiories outside project', default=[])
parser.add_argument('--defines', nargs='*', metavar="MACRO", help='Defined symbols', default=[])
parser.add_argument('--dest', nargs=1, metavar='DEST', help='Project destination folder', default='.')
args = parser.parse_args()

This is a really useful Python utility, which parses script parameters and populates it in args variable set on the bottom. After that, we can access parameter with a human-readable form like args.name[0] (these objects are arrays – that’s why it is indexed with 0.

The next thing to do with our .project file template is to fill linkedResources element. It is not the most important part – linkedResources are directories which are outside project, but they are linked. Not every project contains it, but these paths may be included with --external parameter.

linkedResources = ET.SubElement(root, 'linkedResources')
for ex in args.external:
        opt = ET.SubElement(linkedResources, 'link')
        ET.SubElement(opt, 'name').text = os.path.basename(ex)
        ET.SubElement(opt, 'type').text = '2'
        ET.SubElement(opt, 'location').text = ex

External directories should be passed as space-separated paths. for-loop is passing all of them and creating new subnodes under linkedResources . These are all things configured in .project file. It is written to the project directory (.settings folder is already there).

tree.write(args.dest[0] + '/.project', 'UTF-8', True)

Now it is time to setup .cproject file. As its name says it is related strictly to C language. It contains things like source and header directories.

tree = ET.parse("template/.cproject-stub")
root = tree.getroot()
cdt = root.find(".//*[@moduleId='cdtBuildSystem']")

for incPath in cdt.findall('.//*/option[@valueType="includePath"]'):
        for path in args.includes:
                opt = ET.SubElement(incPath, 'listOptionValue')
                opt.set('builtIn', 'false')
                opt.set('value', os.path.abspath(path))

for defineNode in cdt.findall('.//*/option[@valueType="definedSymbols"]'):
        for define in args.defines:
                opt = ET.SubElement(defineNode, 'listOptionValue')
                opt.set('builtIn', 'false')
                opt.set('value', define)

sourcesNode = cdt.find('.//*/sourceEntries')
for source in args.sources:
        opt = ET.SubElement(sourcesNode, 'entry')
        opt.set('flags', 'VALUE_WORKSPACE_PATH')
        opt.set('kind', 'sourcePath')
        opt.set('name', source)
for ex in args.external:
        opt = ET.SubElement(sourcesNode, 'entry')
        opt.set('flags', 'VALUE_WORKSPACE_PATH|RESOLVED')
        opt.set('kind', 'sourcePath')
        opt.set('name', os.path.basename(ex))

Just like in .project file .cproject-stub is opened, loaded and parsed to a tree structure with root node object root. This time all operations are made on subnodes of storageModule with attribute moduleId="cdtBuildSystem". Under this node, we can find the configuration of all languages added to this project. In Eclipse CDT C/C++ Project it is usually C, C++, and assembly. My script loops on configuration options of all languages (outer for-loop) for the sake of simplicity. These options are includePath, definedSymbols, and sourceEntries. Just like in previous parts all arguments are translated from space-separated list to Eclipse-known XML node list. I’ve marked all of these as builtin symbols/paths even if it’s not true. There is no sense in splitting them. Some attributes and subnodes were reverse-engineered, I think they are self-explaining (like kind sourcePath). Worth noting here is that script links external directories, already configured in .project file as the source path. The idea behind it is that as external paths I gave library source paths, which in turn could be easily accessed during development. It is really useful, but I had problems with naming. When I wrote that script I assumed that the source directory name will distinguish it from other folders. It was a mistake, almost all of the source folders are named src. Fortunately, this script is really simple and you can manage your own naming strategy if it is a problem for you :).

Last, but not least, saving .cproject was a little bit problematic due to not standard XML header. It looks like this:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?fileVersion 4.0.0?>

ElementTree parsed input file well, but there was no way to restore in the same form (using standard API). This is my workaround:

f = open(args.dest[0] + '/.cproject', 'w')
f.write('<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<?fileVersion 4.0.0?>' + ET.tostring(root))
f.close()

The script is ready, it’s full form is presented at the bottom of the article. Now it is time to use it. As I mentioned above, the best place to call it is Makefile script. It is really easy, and beautiful help generated by argparse module is helpful here:

usage: eclipse-generator.py [-h] [--name NAME] [--includes PATH [PATH ...]]
                            [--sources PATH [PATH ...]]
                            [--external [PATH [PATH ...]]]
                            [--defines [MACRO [MACRO ...]]] [--dest DEST]

Eclipse project generator, creates .project and .cproject files

optional arguments:
  -h, --help            show this help message and exit
  --name NAME           Project name, will be displayed in project explorer
  --includes PATH [PATH ...]
                        Paths to include directories, might be relative
  --sources PATH [PATH ...]
                        Paths to source locations, relative to project only!
  --external [PATH [PATH ...]]
                        Source directiories outside project
  --defines [MACRO [MACRO ...]]
                        Defined symbols
  --dest DEST           Project destination folder

Source and includes are straightforward, but what with defines parameter? Without it, perfect indexing C code would be impossible – yes, we have all proper include and source paths, but it usually contains a lot of conditional compilation statements. They are based on a version of the language standard, on POSIX conformance, etc. – all the things which are handled by the toolchain. Luckily, there is an option of gcc, that shows all builtin defined macros. The option of our need is

gcc -dM -E - < /dev/null

-dM option outputs all defines used by the compiler, -E option stops build process on preprocessing, – tell gcc to take code from stdin which is redirected from /dev/null. There is no need to running the compilation process in this one-liner, so it is stopped on preprocessing. We are fetching builtin macros only, so passing C files is not necessary. Without /dev/null redirection, the program would stop waiting for input.

Unfortunately, that’s not all. gcc gives us all #define directives in a C-understood format. That is not what our script wants. But with some sed knowledge we can easily handle this problem:

gcc -dM -E - < /dev/null | cut -c 9- | sed 's/ /=/' | sed 's/ /\\ /g'

First element in pipe – cut – eliminates #definedirective. sed 's/ /=/'substitutes first occurrence of space (after macro name) with = character (this is expected by CDT). Last sed call prefixes all spaces with a backslash – without it, space would be treated as a separator.

With all those things, joining it together in Makefile should be simple, but there is one problem with the last pipelined command. Makefile engine has problems with parentheses in its variables. This is why our command inside Makefile would be not as nice as it was:

LEFTPAREN := (
RIGHTPAREN := )
DEFINES := $(shell gcc -dM -E - < /dev/null | cut -c 9- | sed 's/ /=/' | sed 's/$(LEFTPAREN)/\\$(LEFTPAREN)/g' | sed 's/$(RIGHTPAREN)/\\$(RIGHTPAREN)/g' | sed 's/ /\\ /g')

And that’s all. I hope my script will save you plenty of time on Eclipse configuration, just like in my case. Any enhancement and comments are welcome. Here you can find the whole working script. .settings directory and .(c)project stubs can be easily extracted from bare CDT project.

import xml.etree.ElementTree as ET
import os
import argparse

parser = argparse.ArgumentParser(description='Eclipse project generator, creates .project and .cproject files')
parser.add_argument('--name', nargs=1, metavar='NAME', help='Project name, will be displayed in project explorer')
parser.add_argument('--includes', nargs='+', metavar='PATH', help='Paths to include directories, might be relative')
parser.add_argument('--sources', nargs='+', metavar='PATH', help='Paths to source locations, relative to project only!')
parser.add_argument('--external', nargs='*', metavar='PATH', help='Source directiories outside project', default=[])
parser.add_argument('--defines', nargs='*', metavar="MACRO", help='Defined symbols', default=[])
parser.add_argument('--dest', nargs=1, metavar='DEST', help='Project destination folder', default='.')
args = parser.parse_args()

print("Generating .project file...")

tree = ET.parse("template/.project-stub")
root = tree.getroot()
projectNameNode = root.find('name')
projectNameNode.text = args.name[0]

linkedResources = ET.SubElement(root, 'linkedResources')
for ex in args.external:
	opt = ET.SubElement(linkedResources, 'link')
	ET.SubElement(opt, 'name').text = os.path.basename(ex)
	ET.SubElement(opt, 'type').text = '2'
	ET.SubElement(opt, 'location').text = ex

tree.write(args.dest[0] + '/.project', 'UTF-8', True)

print("Generating .cproject file...")
tree = ET.parse("template/.cproject-stub")
root = tree.getroot()
cdt = root.find(".//*[@moduleId='cdtBuildSystem']")

for incPath in cdt.findall('.//*/option[@valueType="includePath"]'):
	for path in args.includes:
		opt = ET.SubElement(incPath, 'listOptionValue')
		opt.set('builtIn', 'false')
		opt.set('value', os.path.abspath(path))

for defineNode in cdt.findall('.//*/option[@valueType="definedSymbols"]'):
	for define in args.defines:
		opt = ET.SubElement(defineNode, 'listOptionValue')
		opt.set('builtIn', 'false')
		opt.set('value', define)

sourcesNode = cdt.find('.//*/sourceEntries')
for source in args.sources:
	opt = ET.SubElement(sourcesNode, 'entry')
	opt.set('flags', 'VALUE_WORKSPACE_PATH')
	opt.set('kind', 'sourcePath')
	opt.set('name', source)
for ex in args.external:
	opt = ET.SubElement(sourcesNode, 'entry')
	opt.set('flags', 'VALUE_WORKSPACE_PATH|RESOLVED')
	opt.set('kind', 'sourcePath')
	opt.set('name', os.path.basename(ex))

f = open(args.dest[0] + '/.cproject', 'w')
f.write('<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<?fileVersion 4.0.0?>' + ET.tostring(root))
f.close()