Indexing your office documents with Elastic and FSCrawler y o David Pilat Developer | Evangelist, Communit @dadoonet

and getting content and metadata ; { ; ; ; ; ; ; { static void extractTextAndMetadata(InputStream stream) throws Exception BodyContentHandler handler = new BodyContentHandler() Metadata metadata = new Metadata() try (stream) new DefaultParser().parse(stream, handler, metadata, new ParseContext()) String extractedText = handler.toString() String title = metadata.get(TikaCoreProperties.TITLE) String keywords = metadata.get(TikaCoreProperties.KEYWORDS) String author = metadata.get(TikaCoreProperties.CREATOR) } } Parsing a stream

n

ingest-attachment plugi extracting from BASE64 or CBOR

An ingest pipeline

ingest-attachment processor plugin using Tika behind the scene

Demo 8 https://cloud.elastic.co

r FSCrawle You know, for files…

Disclaime This project is a community project. It is not officially supported by Elastic. Support is only provided by FSCrawler community on discuss and stackoverflow.

r http://discuss.elastic.co/ https://stackoverflow.com/questions/tagged/fscrawler

FSCrawle Architecture FSCrawler Local Dir JSON (noop) ES6 Mount Point XML ES7 SSH / SCP Apache Tika HTTP Rest r Inputs Filters Outputs

FSCrawle Key Features • • • Much more formats than ingest attachment plugi OCR (Tesseract Much more metadata than ingest attachment plugin (See https://fscrawler.readthedocs.io/en/latest/admin/fs/elasticsearch.html#generated-fields) n r Language detection ) •

Documentation • • • • https://fscrawler.readthedocs.io/ https://fscrawler.readthedocs.io/en/latest/user/tutorial.html https://fscrawler.readthedocs.io/en/latest/user/formats.html https://fscrawler.readthedocs.io/en/latest/admin/fs/index.html https://fscrawler.readthedocs.io/en/latest/

Demo 15 https://cloud.elastic.co

r FSCrawle even better with a UI

FSCrawle r Workplace Search integration

FSCrawle Architecture FSCrawler Local Dir JSON (noop) ES6 Mount Point XML ES7 SSH / SCP Apache Tika WP7 Filters Outputs HTTP Rest r Inputs

Demo 20 https://cloud.elastic.co

Thanks PR are warmly welcomed! ! https://github.com/dadoonet/fscrawler