python - Elasticsearch delay in store and search immediately -
i using
elasticsearch python. , use dsl driver in python.
my script below.
import time elasticsearch_dsl import doctype, string elasticsearch import exceptions es_exceptions elasticsearch_dsl.connections import connections elasticsearch_index = 'test' class studentdoc(doctype): student_id = string(required=true) tags = string(null_value=[]) class meta: index = elasticsearch_index def save(self, **kwargs): ''' override set metadata id ''' self.meta.id = self.student_id return super(studentdoc, self).save(**kwargs) # define default elasticsearch client connections.create_connection(hosts=['localhost:9200']) # create mappings in elasticsearch studentdoc.init() student_doc_obj = \ studentdoc( student_id=str(1), tags=['test']) try: student_doc_obj.save() except es_exceptions.serializationerror ex: # catch both exception raise elasticsearch logger.error('error while creating elasticsearch data') logger.exception(ex) else: print "*"*80 print "student created:", student_doc_obj print "*"*80 search_docs = \ studentdoc \ .search().query('ids', values=["1"]) try: student_docs = search_docs.execute() except es_exceptions.notfounderror ex: logger.error('unable data elasticsearch') logger.exception(ex) else: print "$"*80 print student_docs print "$"*80 time.sleep(2) search_docs = \ studentdoc \ .search().query('ids', values=["1"]) try: student_docs = search_docs.execute() except es_exceptions.notfounderror ex: logger.error('unable data elasticsearch') logger.exception(ex) else: print "$"*80 print student_docs print "$"*80 in script, creating studentdoc , try access same doc when create. empty response when search on record.
output
******************************************************************************** student created: {'student_id': '1', 'tags': ['test']} ******************************************************************************** $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ <response: []> $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ <response: [{u'student_id': u'1', u'tags': [u'test']}]> $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ save command execute , store data, why search not return tat data. after 2 second sleep, return data. :(
tried same curl commands, same output.
echo "create data" curl http://localhost:9200/test/student_doc/2 -x put -d '{"student_id": "2", "tags": ["test"]}' -h 'content-type: application/json' echo echo "search id" curl http://localhost:9200/test/student_doc/_search -x post -d '{"query": {"ids": {"values": ["2"]}}}' -h 'content-type: application/json' echo is there delay in storing data elasticsearch?
yes, once index new document, not available until refresh of index occurs. have few options, though, main ones being.
a. can refresh test index using underlying connection after saving student_doc_obj , before searching it:
connections.get_connection.indices.refresh(index= elasticsearch_index) b. can get document instead of searching it, get real-time , doesn't need wait refresh:
student_docs = studentdoc.get("1") similarly, using curl, can add refresh query string parameter in put call
echo "create data" curl 'http://localhost:9200/test/student_doc/2?refresh=true' -x put -d '{"student_id": "2", "tags": ["test"]}' -h 'content-type: application/json' or can document id
echo "get id" curl -xget http://localhost:9200/test/student_doc/2
Comments
Post a Comment