[FORUM BUG] Search doesn't work properly

[b_force]

I wanted to use the search to find some topics that has been flooded away by the huge pile of new messages.

This topic was started in August, about one or two weeks ago.
Unfortunately I couldn't find it with the search.
However, I noticed that after using other words, the search doesn't seem to get any posts that are made in Augusts.
Mostly because all I found were posts at the end of July.

To show a few random results;

This topic has a very unique "word" in the title;
https://www.eevblog.com/forum/projects/gps-external-antenna-for-v-kel-vk2828u7g5lf/

If I search for "VK2828U7G5LF" (topic subject only), the result is zero.

Ok, another one, topic started at August 16;
https://www.eevblog.com/forum/projects/old-server-psu-pinout-(-astec-aa21060-)-or-how-to-turn-it-on/
Searching for "AA21060' = no result

Next one on August 2nd
https://www.eevblog.com/forum/projects/problem-with-hmc253/
No result

https://www.eevblog.com/forum/projects/proximity-sensor-anyone-use-mtch101-or-mtch105/
Searching for MTCH101, finally results!!!!


So, something is going wrong here??? 

Also, you can only sort on 'newest topic first', but that's only about when the topic was started.
There is not a option, 'newest post first'.

It's rather frustrating because on this forum people are pooing out new topics like the way bunnies do their magic.
This means that some topics are waaay gone after a week (sometimes even days)

[Fungus]

I wanted to use the search to find some topics that has been flooded away by the huge pile of new messages.

It doesn't work. It's never worked.

It's rather frustrating

Use google instead. It works perfectly.

[sasa]

The most forum engines fails in searching local database.

Fortunately, Google frequently update active public sites/forums. Instead using forum engine, searching by Google gives  exact link:

VK2828U7G5LF site:www.eevblog.com

One of the most helpful options offered by Google.

[Fungus]

The most forum engines fails in searching local database.

Fortunately, Google frequently update active public sites/forums. Instead using forum engine, searching by Google gives  exact link:

VK2828U7G5LF site:www.eevblog.com

One of the most helpful options offered by Google.

Or simply: "eevblog VK2828U7G5LF"

[b_force]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

[Monkeh]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

Well, this isn't phpBB..

[Brumby]

Indeed it's not.

It's SMF ... something which is displayed at the bottom of the page - however, with Dave's choice of background, it's next to impossible to see, unless you select the text.

[b_force]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

Well, this isn't phpBB..

So?? 
What is your point?

I was just replying to the comment that 'most forum engines fail in searching'.
Which isn't true in my experience.


In fact, this is the first one were it doesn't work.
(and no, not all these other forums work on phpBB)

[RoGeorge]

Use Google search instead, and specify to Google that you want to return only the results coming from eevblog.com. Example (type this in Google search):

Code: [Select]

VK2828U7G5LF site:eevblog.com

[Monkeh]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

Well, this isn't phpBB..

So?? 
What is your point?

Your sentence implies that this, like the 'other phpBB' forums, runs phpBB.

[b_force]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

Well, this isn't phpBB..

So?? 
What is your point?

Your sentence implies that this, like the 'other phpBB' forums, runs phpBB.

I really don't see that implication at all?
Maybe I should have used brackets.
Doesn't matter, also not very relevant to be very honest.

The message was, other forums can do it, so why is it such a struggle here?

[edavid]

The message was, other forums can do it, so why is it such a struggle here?

Because this forum uses SMF, and SMF search doesn't work very well.

You could read about it in the many previous threads on the subject, except you probably can't find them 

[sasa]

I was just replying to the comment that 'most forum engines fail in searching'.
Which isn't true in my experience.

I had quite a problem in the past to search on any forum and what is wrote is based on my experience. Probably latest versions indeed use FTS feature of MySQL or whatever database engine is used. Without FTS feature, indeed, it is not easy or it is rather slow to accomplish search. However, with FTS feature, database become really huge...

Anyway, it is not at all important is it phpBB, SMF or anything else. It is easy fixable by Google search site feature.

I do not use forums at all, though, as any thread ends as this one searching needle in a haystack...

[Brumby]

The most forum engines fails in searching local database.

I don't know, but most other phpBB forums I know, are doing pretty fine actually.
Never had issues, except this forum.

Well, this isn't phpBB..

So?? 
What is your point?

Your sentence implies that this, like the 'other phpBB' forums, runs phpBB.

I really don't see that implication at all?

Initially, I didn't either - but then I didn't know about "phpBB" being one of the players.  (I've never really paid much attention to the software different forums run - but I do recognise SMF.)

Once I had Googled "phpBB", I got the same impression as Monkeh.

[Brumby]

But, yeah, I've not had a brilliant search experience with SMF software.

[amyk]

Use Google search instead, and specify to Google that you want to return only the results coming from eevblog.com. Example (type this in Google search):

Code: [Select]

VK2828U7G5LF site:eevblog.com

Google search is broken too, unfortunately. It picked up that thread but as of this post, this query...

Code: [Select]

site:eevblog.com hawkins electrical guide...doesn't find this thread of mine that's been around for almost a year. Ironically, the forum search does find that one.

[Electro Detective]

Hi guys, hopefully not too Off Topique or ranty   

every time I try to view this awesome IR thermal camera post by Fraser it bogs down my browser to a halt till I close the page

https://www.eevblog.com/forum/thermal-imaging/thermal-camera-related-videos-and-reviews/msg1220335/#msg1220335

Fraser's other related posts load fine but not this one, have tried a few times    and given up


Please advise if the problem is the page or the browser, wasn't sure where to submit this query     

Thanks    

[PA0PBZ]

Please advise if the problem is the page or the browser, wasn't sure where to submit this query     

Loads fine for me in Chrome, Firefox, Edge and IE.
(after manually copying the complete link)

[Electro Detective]

Please advise if the problem is the page or the browser, wasn't sure where to submit this query     

Loads fine for me in Chrome, Firefox, Edge and IE.
(after manually copying the complete link)

PA0PBZ, thanks for checking     and apologies are in order as unfortunately I pasted the wrong link      and updated my previous comment

Here is the suspect one >  https://www.eevblog.com/forum/thermal-imaging/thermal-camera-related-videos-and-reviews/msg1220335/#msg1220335

[EEVblog]

I looked at the forum search settings and this is strange:



I have no idea what SphinxQL is is or when that was set. Maybe that's the problem.
I have changed to "Fulltext index", see if that makes a difference.
I am also building a custom "full" index but it will take some time.

[EEVblog]

This topic has a very unique "word" in the title;
https://www.eevblog.com/forum/projects/gps-external-antenna-for-v-kel-vk2828u7g5lf/

If I search for "VK2828U7G5LF" (topic subject only), the result is zero.

I just tried that and it now works! 

[EEVblog]

And I just tried a search on new thread started yesterday by searching "Vocabulary for beginners" and it found it as the only match as expected.
So the current full index seems to be up to date and being added to.
I think the search problem could be solved....

[edavid]

And I just tried a search on new thread started yesterday by searching "Vocabulary for beginners" and it found it as the only match as expected.
So the current full index seems to be up to date and being added to.
I think the search problem could be solved....

Thanks!  It is definitely working better.

Are there any options for the default order of search results?  Or any way to display a "sort by" box, or "search within results"?

[gnif]

I looked at the forum search settings and this is strange:



I have no idea what SphinxQL is is or when that was set. Maybe that's the problem.
I have changed to "Fulltext index", see if that makes a difference.
I am also building a custom "full" index but it will take some time.

I hope you didn't turn this off, we switched to sphinx for full text indexing. If its not working we need to fix it, not turn it off.

[gnif]

It seems that somehow the cronjob that updates the sphinx index has been disabled/removed, likely by cPanel

Edit: I have switched Sphinx back on and will investigate the issue. There is simply too much content to rely on MySQL to search the content which is why we moved to a full text search engine (Sphinx)
Edit2: Issue resolved, the index has been rebuilt and the search is now working as expected.

[EEVblog]

I hope you didn't turn this off, we switched to sphinx for full text indexing. If its not working we need to fix it, not turn it off.

Yep, sorry, I switched back to Full Index.
Seems to work fine though?
Why did we switch to Sphinx? and how can we be sure it's not going to get disabled again and no one notice until the search gets bad enough?

[gnif]

I hope you didn't turn this off, we switched to sphinx for full text indexing. If its not working we need to fix it, not turn it off.

Yep, sorry, I switched back to Full Index.
Seems to work fine though?
Why did we switch to Sphinx? and how can we be sure it's not going to get disabled again and no one notice until the search gets bad enough?

MySQL is too slow when several searches occur at once, we saw widespread issues when Bing was using the search to index the website, but also when I tested it with a few concurrent searches. MySQL is not designed for this level of search/index, here are the stats from Sphinx:

Code: [Select]

collected 1276001 docs, 865.5 MB
sorted 143.9 Mhits, 100.0% done
total 1276001 docs, 865542064 bytes

So there is 143.9 million search terms being indexed. Services like google, ebay, amazon, etc... all use a full text search index such as Sphinx for datasets like this.

To avoid cPanel messing with it again I have created a system account for sphinx and configured a crontab for it, cPanel will not touch this account as it knows nothing about it.

Some more info on Sphinx (from http://sphinxsearch.com/about/sphinx/):

• Indexing performance. Sphinx indexes up to 10-15 MB of text per second per single CPU core, that is 60+ MB/sec per server (on a dedicated indexing machine).
• Searching performance. Searching through 1,000,000-document, 1.2 GB text collection that we use for everyday development and testing runs at 500+ queries/sec on a 2-core desktop machine with 2 GB of RAM.
• Scalability. Biggest known Sphinx cluster indexes 25+ billion documents, resulting in over 9TB of data. Busiest known one is Craigslist, serving 300+ million search queries/day.

[EEVblog]

Thanks gnif
It's switched back to Sphinx and seems to work again.

[PA0PBZ]

PA0PBZ, thanks for checking     and apologies are in order as unfortunately I pasted the wrong link      and updated my previous comment

Here is the suspect one >  https://www.eevblog.com/forum/thermal-imaging/thermal-camera-related-videos-and-reviews/msg1220335/#msg1220335

For me it doesn't halt but depending on the browser used it takes about 15-30 seconds to embed all 36 YouTube links. Of course if you run out of memory or your connection is slow anything can happen.

[gnif]

PA0PBZ, thanks for checking     and apologies are in order as unfortunately I pasted the wrong link      and updated my previous comment

Here is the suspect one >  https://www.eevblog.com/forum/thermal-imaging/thermal-camera-related-videos-and-reviews/msg1220335/#msg1220335

For me it doesn't halt but depending on the browser used it takes about 15-30 seconds to embed all 36 YouTube links. Of course if you run out of memory or your connection is slow anything can happen.

Confirmed, slow load of this page is not server related but embedded content related.

[Electro Detective]

Thanks guys      that page nukes my browser back to the Ice Age 

[EEVblog]

Thanks guys      that page nukes my browser back to the Ice Age 

Took about 10 seconds to load that on my Surface Pro Win 10 Chrome with 30 other tabs open (yes, I have a Tab sickness), but no problems.

[b_force]

This topic has a very unique "word" in the title;
https://www.eevblog.com/forum/projects/gps-external-antenna-for-v-kel-vk2828u7g5lf/

If I search for "VK2828U7G5LF" (topic subject only), the result is zero.

I just tried that and it now works! 

Thanks!
 

Would it also be possible to search on 'newest post/message first'?
Now you can only select newest topic first (which is not the same)