SyntaxHighlighter

Friday, February 12, 2010

Thinking Sphinx installation, indexing, deploying on Engine Yard

This took a few days, but it's very simple to do and should take minutes. If you're using Engine Yard Cloud, it's not obvious, but it is trivial. The post assumes you have Thinking Sphinx and the Sphinx daemon running on your dev machine and aren't new to either. I used the Thinking Sphinx plugin, as Engine Yard recommends, but I see there is also a gem which my work.

I'm working from memory here, so please let me know if you find any details missing or unclear.

Here's the steps I took:

First you need to clone the engineyard github repository for their chef recipes. This is how engine yard knows you want to have the sphinx daemon running.

I cloned this to my app root, but it probably didn't matter. You don't check it in.

http://github.com/engineyard/ey-cloud-recipes

Once you have this, you edit line 10 in the cookbooks/thinking_sphinx/recipes/default.rb file.

10. run_for_app("app_name") do |app_name, data|

Just add your engine yard app_name here.

Then follow the Installation instructions on the bottom of that page.

Indexing
You'll want to index regularly. For this, I set up a cron job.

cd /data/app_name/current && rake ts:index RAILS_ENV=production

Note: this didn't work for me at first. Not sure why. It could have been a spelling mistake or maybe it had something to do with how I was setting up the deployment. I'll describe deployment next.

Deployment
When you deploy on Engine Yard, you need to stop, index, and start thinking sphinx. To do this, you need to make a "deploy" directory under your rails app_root. For me, I have one file in there "after_restart.rb" and the contents of this file is just this:

sudo "cd #{current_path} && rake ts:stop RAILS_ENV=production"
sudo "cd #{current_path} && rake ts:index RAILS_ENV=production"
sudo "cd #{current_path} && rake ts:start RAILS_ENV=production"

Note: this also didn't work for me right away. I was getting an error that said it could not cd because the directory didn't exist, but I stopped seeing that error. Not sure why this happened.

More information
You may have done some searching to figure around this issue already. When I searched, I found there was a lot of extra or mis- information that made it tough to figure out what to do in production. There are many documents on Engine Yard about this, including the following:

http://www.engineyard.com/blog/2009/5-tips-for-sphinx-indexing/
https://cloud-support.engineyard.com/discussions/questions/265-thinking-sphinx

But these documents all seem to be out of date, at least for a Cloud deployment. Please read these to see if they apply to your needs, but they seem to contain a lot of information that was either old, or did not apply to the standard deployment.

Engine Yard support
When I wrote Engine Yard to ask for a clarification to the online docs, they said that I could submit a ticket to get an answer (this requires the $200+ per month service). It really seems like this should have been a quick answer for them, in the knowledge base somewhere, or worthy of a comment in their online docs. I didn't end up buying the support package, but may in the future.

Also, I posted a question on the community support forum, not the cloud support forum. I didn't realize these were different. The UI is identical and I'm not really sure how I got to the wrong site. Keep that in mind if you're asking for help in the forums.

Anyway, hope this helps someone save some time.

3 comments:

  1. Mark,

    Thank you for the detailed write up of the steps you took to get sphinx running for you. I'm sure it will help others out there in the community.

    I wanted to apologize for the confusion regarding the two support forums. Initially the division was for "Public Cloud" and "Private Cloud" clients. We plan to merge them into one.

    Regarding online docs, you should keep your eye open for some big news on our blog. We've got some big plans for documentation that is easy to find, up-to-date and free to the community.

    ReplyDelete
  2. @filmprog - i am also a EY AppCloud customer looking for more documentation on getting sphinx set up. any update on the big news you mentioned? Thanks.

    ReplyDelete
  3. I could use some help on this as well. The out-of-date documentation is quite frustrating.

    ReplyDelete