Author Topic: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)  (Read 13906 times)

0 Members and 1 Guest are viewing this topic.

Offline Mark Hennessy

  • Frequent Contributor
  • **
  • Posts: 305
  • Country: gb
    • My electronics and audio website
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #25 on: December 29, 2013, 03:35:24 pm »
Are you sure? As far as I can understand - and from how it's explained on that thread I linked to - it's simply using Google Custom Search.

OK - because I've been known to get things wrong, I thought I'd better check this out myself.

I copied and pasted the code I quoted earlier into a page on my website, and found that it did indeed work. Anything I typed in returned a page of results from the UK vintage forum. That proved that Google isn't simply looking at the referring site before deciding which site is raising the query...

Next, that funny business in the "cx" field:

I changed a single character in that string, and that stopped the search from working. Good - as suspected, the long string in the "cx" field is indeed telling Google which site is being searched.

So I decided to set up a Custom Search, using the URL provided by gmb42 earlier. That was dead-easy - literally just a case of typing in my site URL and clicking on "go". In return, I have been given a load of stuff that I can paste into my website to generate the Google search:

Code: [Select]
<script>
  (function() {
    var cx = '005351797449373264470:qjxoqys2r60';
    var gcse = document.createElement('script');
    gcse.type = 'text/javascript';
    gcse.async = true;
    gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') 
        '//www.google.com/cse/cse.js?cx='   cx;
    var s = document.getElementsByTagName('script')[0];
    s.parentNode.insertBefore(gcse, s);
  })();
</script>
<gcse:search></gcse:search>

But, this all looks a bit complex, so I haven't tried that yet. Instead, I took the HTML that I posted earlier, and simply replaced the "cx" value with the one given to me in the above code. And it works!

All of that took about 5 minutes. Adding something like this to the forum ought to be dead simple - it's just a case of understanding how SMF organise their files/templates/whatever, so that you know which file needs to receive the HMTL to put the search box onto every page...

Good luck!

Mark
 

Offline diynoise

  • Contributor
  • Posts: 12
  • Country: it
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #26 on: December 29, 2013, 03:52:25 pm »
I think that to install that simple HTML code on a smf forum you need a mod, though. At least for it to not cause problems when you update the forum, otherwise you could just edit the theme.

But don't quote me on that, that's what I remember from when I was an admin at a smaller smf forum.
 

Offline Mark Hennessy

  • Frequent Contributor
  • **
  • Posts: 305
  • Country: gb
    • My electronics and audio website
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #27 on: December 29, 2013, 04:02:14 pm »
Can't speak for SMF, but I've done this sort of thing on MyBB and it was trivially easy. Obviously you're at the mercy of the people who code SMF - it's their ecosystem - but I can't believe it's that difficult to anyone who understands PHP. If I get a bit of spare time later I'll download it and play with it...
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 7648
  • Country: us
  • hp>Agilent>Keysight>???
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #28 on: December 30, 2013, 01:21:33 pm »
Has an Admin looked into what I said below? The search only returns 30 results and that's not the way it's 'posed to work.

Methinks the SMF search function is still not set up right. I searched for transistor, then resistor, then capacitor, and I always get back exactly 30 results. That's not coincidence. Should be a whole bunch 'o results for those terms.

Dave, could you check this -

Admin - forum - search - settings

Number of search results per page - ?
Maximum results to show - ?
(0: no limit)
I told my friends I could teach them to be funny, but they all just laughed at me.
 

Offline van-c

  • Regular Contributor
  • *
  • Posts: 69
  • Country: us
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #29 on: December 30, 2013, 04:40:30 pm »
...The search only returns 30 results and that's not the way it's 'posed to work.

I've mentioned this 30 results limit too.   When you're searching for a specific case of a very common topic, having all occurrences returned is crucial, even when the number returned is very large.  I do like the way all cases of the search string are highlighted, though.  The way results are returned by the forum search tool makes finding a particular item of interest embedded in a long list of results a lot easier, IMO, than trudging through many Google results.

If the tool would just return all occurrences without limiting the number, that would help a great deal.
« Last Edit: December 30, 2013, 05:36:34 pm by van-c »
 

Lurch

  • Guest
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #30 on: January 01, 2014, 03:31:03 pm »
This thread is in the wrong place.
 

Offline gnif

  • Administrator
  • *****
  • Posts: 1704
  • Country: au
  • Views and opinions are my own
    • AMD
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #31 on: January 02, 2014, 02:15:56 pm »
It might be the search method that's set up. I had a problem with that on my forum and fixed it by going to a full text index. The Admin here could look into it.


Forum - Search method

Search index

Fulltext index


This should be profiled before just enabling it, this adds 'FULLTEXT' indexes into the database which can hurt performance badly with forums as large as this. Also some of the database tables have been converted to InnoDB to correct performance issues we were having some time ago, InnoDB does not support FULLTEXT indexes (yet). A better option would be to look at a proper dedicated indexer such as Sphynx, I would be surprised if someone has not made a Sphynx plugin for SMF. Failing that, Google is your friend.
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 7648
  • Country: us
  • hp>Agilent>Keysight>???
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #32 on: January 02, 2014, 02:41:46 pm »
Gnif, the real issue is that the search returns only 30 results, when it is obvious, such as searching for "transistor" or "resistor" that there are far more than 30 occurrences of that text in the database. We were simply troubleshooting that issue to see if anything had an effect on that. Can you offer any insight to the crux of the issue, as I've just now explained? I have been doing some investigating on the SMF forum but have not found the definitive answer yet.

Oh and yes, this thread is in need of splitting.  :)
I told my friends I could teach them to be funny, but they all just laughed at me.
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 16362
  • Country: za
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #33 on: January 02, 2014, 02:42:33 pm »
How about changing the search to use Google in the top bar, with as interim window warning that you are going to the external site. line could still use the default method.
 

Offline gnif

  • Administrator
  • *****
  • Posts: 1704
  • Country: au
  • Views and opinions are my own
    • AMD
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #34 on: January 02, 2014, 02:54:10 pm »
Gnif, the real issue is that the search returns only 30 results, when it is obvious, such as searching for "transistor" or "resistor" that there are far more than 30 occurrences of that text in the database. We were simply troubleshooting that issue to see if anything had an effect on that. Can you offer any insight to the crux of the issue, as I've just now explained? I have been doing some investigating on the SMF forum but have not found the definitive answer yet.

Oh and yes, this thread is in need of splitting.  :)

The reason the number of search results are limited is to prevent overloading of the server, text searches are very processor and I/O intensive... if it was allowed to return everything in one go, a single user could effectively DOS the forums simply by performing a few searches. I am looking at Sphynx, seems that SMF has support for it, I need to set-up a test server and have a play and see how it performs before even thinking of deploying it. I will discuss this with Dave over the coming days, I may even take a read-only copy of these forums as a good test sample set and give a few people access to trial the search functionality.
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 7648
  • Country: us
  • hp>Agilent>Keysight>???
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #35 on: January 02, 2014, 02:57:34 pm »
The reason the number of search results are limited is to prevent overloading of the server, text searches are very processor and I/O intensive...

Oh OK, so you are doing it on purpose. Well nobody said that until now. From what Dave was seeming to say when we were playing with it, he didn't think the 30 result thing was acting properly either. So that's where the confusion lies. I'm not sure that it can't really handle returning more than 30, such as perhaps 100 or so, but it's your call.
I told my friends I could teach them to be funny, but they all just laughed at me.
 

Offline gnif

  • Administrator
  • *****
  • Posts: 1704
  • Country: au
  • Views and opinions are my own
    • AMD
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #36 on: January 02, 2014, 03:01:33 pm »
The reason the number of search results are limited is to prevent overloading of the server, text searches are very processor and I/O intensive...

Oh OK, so you are doing it on purpose. Well nobody said that until now. From what Dave was seeming to say when we were playing with it, he didn't think the 30 result thing was acting properly either. So that's where the confusion lies. I'm not sure that it can't really handle returning more than 30, such as perhaps 100 or so, but it's your call.

I did not set the limit, it is whatever Dave has configured it at, I am not sure why 30 was chosen, could just be the default. The better solution though would be to remove the limit completely and move to a proper indexer as this solution is both scalable and future proof (read: wont cause a slowdown as the forum gets bigger and bigger)
 

Offline marmad

  • Super Contributor
  • ***
  • Posts: 2979
  • Country: aq
    • DaysAlive
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #37 on: January 02, 2014, 03:04:34 pm »
The reason the number of search results are limited is to prevent overloading of the server, text searches are very processor and I/O intensive... if it was allowed to return everything in one go, a single user could effectively DOS the forums simply by performing a few searches. I am looking at Sphynx, seems that SMF has support for it, I need to set-up a test server and have a play and see how it performs before even thinking of deploying it. I will discuss this with Dave over the coming days, I may even take a read-only copy of these forums as a good test sample set and give a few people access to trial the search functionality.

But this is assuming it's working in some logical way - but as already reported, it isn't. When you use advanced search with no restrictions, you get back a single page of 30 results. When you use advanced search with "Search in topic subjects only" (ostensibly limiting the search), you get back multiple pages of 30 results.
 

Offline gnif

  • Administrator
  • *****
  • Posts: 1704
  • Country: au
  • Views and opinions are my own
    • AMD
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #38 on: January 02, 2014, 03:06:34 pm »
The reason the number of search results are limited is to prevent overloading of the server, text searches are very processor and I/O intensive... if it was allowed to return everything in one go, a single user could effectively DOS the forums simply by performing a few searches. I am looking at Sphynx, seems that SMF has support for it, I need to set-up a test server and have a play and see how it performs before even thinking of deploying it. I will discuss this with Dave over the coming days, I may even take a read-only copy of these forums as a good test sample set and give a few people access to trial the search functionality.

But this is assuming it's working in some logical way - but as already reported, it isn't. When you use advanced search with no restrictions, you get back a single page of 30 results. When you use advanced search with "Search in topic subjects only" (ostensibly limiting the search), you get back multiple pages of 30 results.

Thus the reason for moving to a more advanced search engine... the SMF simple search is just too primitive for the size of this forum now.
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 7648
  • Country: us
  • hp>Agilent>Keysight>???
Re: NEWBIES, YOU'RE SCREWED (AKA, Forum Search doesn't work half the time)
« Reply #39 on: January 02, 2014, 03:58:05 pm »
Thus the reason for moving to a more advanced search engine... the SMF simple search is just too primitive for the size of this forum now.

OK gnif that's cool, but I hope you can do that. If you want this to be a valuable resource of technical info (and it is) then people need to perform research with a good return of results. The answer they want might be search result 31 but never to be seen ...

Good Luck.
I told my friends I could teach them to be funny, but they all just laughed at me.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf