Indexer ses documents bureautique avec la suite Elastic et FSCrawler
A presentation at Webinaire Sedona by David Pilato
Indexing your office documents with Elastic and FSCrawler y o David Pilat Developer | Evangelist, Communit @dadoonet
and getting content and metadata ; { ; ; ; ; ; ; { static void extractTextAndMetadata(InputStream stream) throws Exception BodyContentHandler handler = new BodyContentHandler() Metadata metadata = new Metadata() try (stream) new DefaultParser().parse(stream, handler, metadata, new ParseContext()) String extractedText = handler.toString() String title = metadata.get(TikaCoreProperties.TITLE) String keywords = metadata.get(TikaCoreProperties.KEYWORDS) String author = metadata.get(TikaCoreProperties.CREATOR) } } Parsing a stream
n
ingest-attachment plugi extracting from BASE64 or CBOR
An ingest pipeline
ingest-attachment processor plugin using Tika behind the scene
Demo https://cloud.elastic.co
r FSCrawle You know, for files…
Disclaime This project is a community project. It is not officially supported by Elastic. Support is only provided by FSCrawler community on discuss and stackoverflow.
r http://discuss.elastic.co/ https://stackoverflow.com/questions/tagged/fscrawler
FSCrawle Architecture FSCrawler Local Dir JSON (noop) ES6 Mount Point XML ES7 SSH / SCP Apache Tika HTTP Rest r Inputs Filters Outputs
FSCrawle Key Features • • • Much more formats than ingest attachment plugi OCR (Tesseract Much more metadata than ingest attachment plugin (See https://fscrawler.readthedocs.io/en/latest/admin/fs/elasticsearch.html#generated-fields) n r Language detection ) •
Documentation • • • • https://fscrawler.readthedocs.io/ https://fscrawler.readthedocs.io/en/latest/user/tutorial.html https://fscrawler.readthedocs.io/en/latest/user/formats.html https://fscrawler.readthedocs.io/en/latest/admin/fs/index.html https://fscrawler.readthedocs.io/en/latest/
r FSCrawle even better with a UI
FSCrawle r Workplace Search integration
FSCrawle Architecture FSCrawler Local Dir JSON (noop) ES6 Mount Point XML ES7 SSH / SCP Apache Tika WP7 Filters Outputs HTTP Rest r Inputs
Demo 18 https://cloud.elastic.co
Thanks PR are warmly welcomed! ! https://github.com/dadoonet/fscrawler
Vous avez sous la main des tonnes de documents Open Office, Microsoft Office, PDF voire des images… Et vous aimeriez être capable de chercher dans leurs meta-données et dans le contenu lui-même.
Comment faire ? Surtout depuis l’annonce de la fin de Google Search Appliance.
Dans cette session, David expliquera comment Apache Tika peut fournir ce service et comment combiner cette fantastique librairie avec elasticsearch :
- Elasticsearch ingest-attachment plugin
- FSCrawler
- Connecteur Workplace Search pour FSCrawler afin de disposer sur étagère d’une interface utilisateur puissante pour vos documents
Video
Resources
The following resources were mentioned during the presentation or are useful additional information.
Code
The following code examples from the presentation can be tried out live.
Buzz and feedback
Here’s what was said about this presentation on social media.
