9d77619afc
Every once in a while we encounter projects using some custom file extension for files that we could in principle extract, but since the extractor doesn't know about the extension the files are skipped. To handle this, the legacy extractor has a `--file-type` option that one can use to specify a file type to use for all files in that particular extraction. So far, `AutoBuild` has nothing of the sort. This PR proposes to introduce an environment variable `LGTM_INDEX_FILETYPES` to allow a similar customisation. In the fullness of time, this variable would be set through `lgtm.yml` in the usual way, but for now it is undocumented and for internal use only. Specifically, `LGTM_INDEX_FILETYPES` is a newline-separated list of ".extension:filetype" pairs, specifying that files with the given `.extension` should be extracted as type `filetype`, where `filetype` is one of `js`, `html`, `json`, `typescript` or `yaml`. For example, `.jsm:js` causes all `.jsm` files to be extracted as JavaScript. This can also be used to override default file types: for example, by specifying `.js:typescript` all JavaScript files will be extracted as TypeScript.
JavaScript extractor
This directory contains the source code of the JavaScript extractor. The extractor depends on various libraries that are not currently bundled with the source code, so at present it cannot be built in isolation.
The extractor consists of a parser for the latest version of ECMAScript, including a few proposed and historic extensions (see src/com/semmle/jcorn), classes for representing JavaScript and TypeScript ASTs (src/com/semmle/js/ast and src/com/semmle/ts/ast), and various other bits of functionality. Historically, the main entry point of the JavaScript extractor has been com.semmle.js.extractor.Main. However, this class is slowly being phased out in favour of com.semmle.js.extractor.AutoBuild, which is the entry point used by LGTM.
License
Like the LGTM queries, the JavaScript extractor is licensed under Apache License 2.0 by Semmle.