lucene - Stopwords not getting removed - solr -

-

i new using solr , have defined following schema:

<schema name="example" version="1.5"> <fields>     <field name="nodeid" type="string" indexed="true" stored="true" />     <field name="_root_" type="string" indexed="true" stored="false" />     <field name="datetime" type="string" indexed="true" stored="true"         multivalued="true" />     <field name="epochsecs" type="string" indexed="true" stored="true"                     multivalued="true" />     <field name="subject" type="text_general" indexed="true"         stored="true" />     <field name="body" type="text_general" indexed="true"         stored="true" />     <field name="emailid" type="string" indexed="true"         stored="true" />     <field name="compliantflag" type="boolean" indexed="true"                     stored="true" />     <field name="_version_" type="long" indexed="true" stored="true" />     <field name="text" type="text_general" indexed="true" stored="false"         multivalued="true" />     <field name="ngrams" type="myngram" indexed="true" stored="false" required="false" />   </fields> <uniquekey>nodeid</uniquekey> <copyfield source="datetime" dest="text" /> <copyfield source="epochsecs" dest="text" /> <copyfield source="subject" dest="text" /> <copyfield source="body" dest="text" /> <copyfield source="emailid" dest="text" /> <copyfield source="compliantflag" dest="text" /> <copyfield source="text" dest="ngrams"/>  <types>     <fieldtype name="string" class="solr.strfield"         sortmissinglast="true" omitnorms="true"/>     <fieldtype name="long" class="solr.trielongfield"                     precisionstep="0" positionincrementgap="0" />     <fieldtype name="boolean" class="solr.boolfield" sortmissinglast="true"/>     <fieldtype name="text_general" class="solr.textfield"         positionincrementgap="100">         <analyzer type="index">             <tokenizer class="solr.standardtokenizerfactory" />             <filter class="solr.stopfilterfactory" ignorecase="true" words="lang/stopwords_en.txt" />             <filter class="solr.porterstemfilterfactory"/>         </analyzer>         <analyzer type="query">             <tokenizer class="solr.standardtokenizerfactory" />             <filter class="solr.stopfilterfactory" ignorecase="true" words="stopwords.txt" />             <filter class="solr.synonymfilterfactory" synonyms="synonyms.txt" ignorecase="true" expand="true"/>             <filter class="solr.porterstemfilterfactory"/>         </analyzer>     </fieldtype>     <fieldtype name="myngram" stored="false" class="solr.textfield">          <analyzer type="index">              <tokenizer class="solr.standardtokenizerfactory"/>             <filter class="solr.lowercasefilterfactory"/>              <filter class="solr.ngramfilterfactory" mingramsize="2" maxgramsize="5"/>          </analyzer>      </fieldtype> </types>

the stopwords not getting removed "body" field when indexed.

also, how remove special characters \n below field using solr's analysers:

\n \n\n\nthese numbers smurfit has.  \n\np

any appreciated. thanks.

standardtokenizer should create tokens around newlines, spaces, etc., , stopword filter looks, @ glance, should working correctly. should include lowercasefilter above stopwordfilter, prevent matches being case sensitive, though.

i wonder if pertinent question might be: mean "removed"? analysis affects indexed representation of field. not affect stored version retrieve index in way. meant facilitate searching, not transform stored version of text. if remove word "the" through filter, should no longer hits on word "the" while searching, still see in when retrieve document index.


Comments

Post a Comment

Popular posts from this blog

java - WrongTypeOfReturnValue exception thrown when unit testing using mockito -

-
my test list<person> mylist; @test public void testisvalidperson() { mylist = new arraylist<person>(); mylist.add(new person("tom")); when(persondao.get(person)).thenreturn(mylist); when((persondao.get(person)).isempty()).thenreturn(false);//------exception thrown boolean result = service.isvalid("tom"); assertfalse(result); } method tested: public boolean isvalid(string person){ persondao = new persondao(); person personobj = new person(person); return (persondao.get(person).isempty())?false : true; } exception thrown: org.mockito.exceptions.misusing.wrongtypeofreturnvalue: boolean cannot returned get() get() should return list *** if you're unsure why you're getting above error read on. due nature of syntax above problem might occur because: 1. exception *might* occur in wrongly written multi-threaded tests. please refer mockito faq on limitations of concurrency testing. 2. sp
Read more

php - Magento - Deleted Base url key -

-
i working on cleaning url: http://mydomain.com/shop/shop/deals i going through settings in configuration tab in admin panel on magento , change secure , unsecure base urls remove first base url "shop" web page read: http://mydomain.com/shop/deals so changed unsecure base urls mydomain.com/shop mydomain.com , secure 12.345.67.89/shop 12.345.67.89. wrong thing do. i cannot reload page or admin console. how access these files through ftp , change correctly there?? when make changes in admin panel changes made in database not in file. have make changes in database. 1.go database , select core_config_data table under table find web/unsecure/base_url , web/secure/base_url . make changes here. don't forget clear cache in var/cache folder
Read more

android - How to disable Button if EditText is empty ? -

-
i have edittext , button in application. when button clicked,the text entered in edittext added listview. i want disable button if edittext empty.how ? this code button click imagebutton imb=(imagebutton)findviewbyid(r.id.btn_send); imb.setonclicklistener(new onclicklistener() { @override public void onclick(view arg0) { edittext et = (edittext)findviewbyid(r.id.edittext1); string str = et.gettext().tostring(); web1.add(str); toast.maketext(shoutsingleprogram.this, "you entered...."+str, toast.length_short).show(); adapter1.notifydatasetchanged(); et.settext(""); } }); } how can ? edittext1.addtextchangedlistener(new textwatcher() { @override public void ontextchanged(charsequence s, int start, int before, in
Read more