Life as Clay

Posts Tagged ‘Rails

Paperclip, S3 & Delayed Job in Rails on Heroku – Jan 2012

with 2 comments

Edit: I forgot to mention here that with Paperclip 2.4.5 you have to use the ‘paperclip-aws’ gem in order for Paperclip to work with Amazon’s newer ‘aws-sdk’ gem. That is no longer true with Paperclip 2.5.

I followed this good tutorial on how to push paperclip image processing to the background with delayed_job. My Rails app is deployed to Heroku on the cedar stack. I’ve had problems with the NoMethod errors using delayed_job 3.0.0, so I downgraded to 2.1.4. Also using paperclip version 2.4.5. In the end, I found that I could ditch the struct presented in the aforementioned tutorial and just call handle_asynchronously on my reprocessing method. This is what the codes looks like:


class Image < ActiveRecord::Base
  attr_accessible :profile_pic,
                  :caption,
                  :note,
                  :pic_file_name,
                  :pic_content_type,
                  :pic_file_size,
                  :pic_updated_at,
                  :pic,
                  :pic_attributes,
                  :processing
  
  belongs_to :imageable, :polymorphic => true
  
  # Added for paperclip-aws
  def self.s3_config
      @@s3_config ||= YAML.load(ERB.new(File.read("#{Rails.root}/config/s3.yml")).result)[Rails.env]    
  end
  
  has_attached_file :pic,
                    :styles => { 
                      :large => "500x500>",
                      :thumb => "100x100>", 
                      :tiny => "50>x50>", 
                      :smallest => "24>x24>" },   
                      :default_url => '/:attachment/:style/missing.png',               
                      
                      # Added for paperclip-aws
                      :storage => :aws,
                      :s3_permissions => :authenticated_read,
                      :path => "images/:id/:style/:filename",
                      :s3_credentials => {
                        :access_key_id => self.s3_config['access_key_id'],
                        :secret_access_key => self.s3_config['secret_access_key']
                      },
                      :bucket => self.s3_config['bucket'],
                      :s3_protocol => "https"    
                      
  validates_attachment_content_type :pic, :content_type => [ /^image\/(?:jpeg|gif|png)$/, nil ]
  
  # How to implement on Heroku with processing in the background
  # http://madeofcode.com/posts/42-paperclip-s3-delayed-job-in-rails
  
  # cancel post-processing now, and set flag...
     before_pic_post_process do |image|
       if !image.processing && image.pic_changed?
         image.processing = true
         false # halts processing
       end
     end
     
     # call method from after_save that will be processed in the background
     after_save do |image| 
       if image.processing
         processImageJob(image)
       end
     end
     
     def processImageJob(image)
         image.regenerate_styles!
     end
     handle_asynchronously :processImageJob
     
     # generate styles (downloads original first)
     def regenerate_styles!
       self.pic.reprocess! 
       self.processing = false   
       self.save(:validations => false)
     end
     
     # detect if our source file has changed
     def pic_changed?
       self.pic_file_size_changed? || 
       self.pic_file_name_changed? ||
       self.pic_content_type_changed? || 
       self.pic_updated_at_changed?
     end
                      
end

Advertisements

Written by Clay

January 12, 2012 at 14:29

Posted in Code, Rails, Ruby

Tagged with , , , , ,

Using Colorbox with Rails 3.1

with 2 comments

I’m in the midst of creating a mini-Facebook like Rails app that I can use to track events in my daughter’s life. Part of that includes the ability to attach photos to events. I wanted to use a friendly modal popup for displaying larger versions of photos, so I elected to go with ColorBox. It’s a jQuery plugin that is dead simple to use, especially on static pages.

For the “timeline” page, a dynamic page, I never know whether there will be photos to display and if there are, how they are grouped. The idea is that photos that are attached to the same event should all be part of one gallery.

Here’s what I did…

Anything that can end up on the timeline is a “mark” in my app. I use a partial when iterating through the marks to display photos that may be associated with the mark. Here is that partial:

<% if mark.images.count > 0 %>
	<div class="mark_images">
		<% for image in mark.images %>
				<a href="<%= image.pic(:large) %>" class="gallery_<%= mark.id %>" title="<%= image.caption %>"><%= image_tag(image.pic(:tiny), :title => image.caption, :class => "mark_image") %></a>
		<% end%>
	</div>
<% end %>

As you can see, I’m using thumbnail images as links. The images, by the way, are attached using the Paperclip gem from thoughtbot. The code above sets up the HTML that ColorBox needs in order to display the modal.

The rest of the work comes in javascript. I elected to put the ColorBox javascript at the end of my application.js file. It looks like this:

$(document).ready(function() {
	galleries = [];
	$('a[class^="gallery_"]').each( function() {
		if ($.inArray($(this).attr("class"), galleries) == -1) {
			$(this).colorbox({
				rel: $(this).attr("class"),
				maxWidth: "95%",
				maxHeight: "95%"
			});
			galleries.push($(this).attr("class"));
		}
	});
});

Here I create an empty array called galleries. I then find all link elements on the page that have a class that begins with “gallery_“. I iterate on each of them. If the class of the element is not in the galleries array, then I set up ColorBox for that class (gallery) and I add the class name to the galleries array so that I know not to set it up again.

Finally, I use the class of the gallery to relate the images that share the same class together, so that I get the navigational buttons on the modal viewbox.

To make this all work, you have to drop the ColorBox .js file into your assets/javascripts directory and you need to include both the .css file that you prefer (from the ColorBox downloadable examples) and the images that accompany it. I chose to use example 3. I had to modify the css image paths so that it would locate the ColorBox images in my assets/images directory. I elected to rename the enclosing folder to colorbox_images. The css file I ended up with looks like this:

/*
    ColorBox Core Style:
    The following CSS is consistent between example themes and should not be altered.
*/
#colorbox, #cboxOverlay, #cboxWrapper{position:absolute; top:0; left:0; z-index:9999; overflow:hidden;}
#cboxOverlay{position:fixed; width:100%; height:100%;}
#cboxMiddleLeft, #cboxBottomLeft{clear:left;}
#cboxContent{position:relative;}
#cboxLoadedContent{overflow:auto;}
#cboxTitle{margin:0;}
#cboxLoadingOverlay, #cboxLoadingGraphic{position:absolute; top:0; left:0; width:100%;}
#cboxPrevious, #cboxNext, #cboxClose, #cboxSlideshow{cursor:pointer;}
.cboxPhoto{float:left; margin:auto; border:0; display:block;}
.cboxIframe{width:100%; height:100%; display:block; border:0;}

/*
    User Style:
    Change the following styles to modify the appearance of ColorBox.  They are
    ordered & tabbed in a way that represents the nesting of the generated HTML.
*/
#cboxOverlay{background:#000;}
#colorbox{}
    #cboxContent{margin-top:20px;margin-bottom:20px}
        .cboxIframe{background:#fff;}
        #cboxError{padding:50px; border:1px solid #ccc;}
        #cboxLoadedContent{border:5px solid #000; background:#fff;}
        #cboxTitle{position:absolute; bottom:-15px; left:0; color:#ccc;}
        #cboxCurrent{position:absolute; top:-20px; right:0px; color:#ccc;}
        #cboxSlideshow{position:absolute; top:-20px; right:90px; color:#fff;}
        #cboxPrevious{position:absolute; top:50%; left:5px; margin-top:-32px; background:url(/assets/colorbox_images/controls.png) no-repeat top left; width:28px; height:65px; text-indent:-9999px;}
        #cboxPrevious:hover{background-position:bottom left;}
        #cboxNext{position:absolute; top:50%; right:5px; margin-top:-32px; background:url(/assets/colorbox_images/controls.png) no-repeat top right; width:28px; height:65px; text-indent:-9999px;}
        #cboxNext:hover{background-position:bottom right;}
        #cboxLoadingOverlay{background:#000;}
        #cboxLoadingGraphic{background:url(/assets/colorbox_images/loading.gif) no-repeat center center;}
        #cboxClose{position:absolute; top:5px; right:5px; display:block; background:url(/assets/colorbox_images/controls.png) no-repeat top center; width:38px; height:19px; text-indent:-9999px;}
        #cboxClose:hover{background-position:bottom center;}

I also chose to move the title to the bottom of the image because that way, it doesn’t bump up against the image count when viewed on a mobile device. That required setting a bottom margin for #cboxContent and changing this line: #cboxTitle{position:absolute; bottom:-15px; left:0; color:#ccc;}.

There are a variety of other options that can be used with the javascript call to alter the modal dialogue. It’s worth taking a look at the ColorBox page for some examples.

Overall, I’m really happy with this solution and probably will use it again in the future. If you’re interested in the app that I’m developing, you can find it on github, called Blytheline. It’s an early work in progress, so it’s a bit rough around the edges!

Written by Clay

November 29, 2011 at 12:53

Returning / Dragging Myself to Rails 3.1

leave a comment »

I’ve been away for a few months, on paternity leave. It was a delight. It now is my turn to return to work and pick up where I left off. You probably know that I work with Rails sometimes if you read this blog. The last time I built a large Rails app was just prior to the 3.0 release. I return to find 3.1 on rc5 and I decided it would be a good idea to get up to speed with it.

Rails 3.1 is a huge departure from Rails 2.x. I feel lost — almost as if I have to learn an entirely new framework! Between updating gems, rails, rack, etc., I decided that I had better start with a new gemset. After stumbling through an RVM upgrade last night (stumbling because I allowed too many versions to pass in-between and the upgrade was far from smooth), I installed 3.1.0.rc5 and started a test app. I ran into a few bugs related to the rc status of the release. The first is that I was receiving a JSON error on a vanilla blank app. If you bump into that, just require these gems:

gem 'multi_json'
gem 'json'

The second is that there was a problem with the arel version. Including the following in the Gemfile fixed the issue for the time being:

gem 'arel', '2.1.4' # remove when 3.1.0 is released

The app boots without errors now, so I’m off to plug away at the new Asset Pipeline and other features. If you’re starting fresh with Rails 3.1, check out this page at Github for a great tutorial on how to get started.

Before I go, however, I want to point to a lovely bit of Processing work by Justin Livi. He created a sketch that randomly generates dandelions like the one below. Check out his site to see the interactive applet.

Written by Clay

August 11, 2011 at 08:34

Posted in Rails

Tagged with ,

Using Nokogiri and Ruby on Rails to grab citations from PubMed

with 12 comments

Update: I put a working example up on github (Rails 3.0.4 and Nokogiri 1.4.4): https://github.com/ccheaton/pubnoko

##############################

Following on the last post… I needed to provide to a client a custom database of PubMed citations, links, datasets, and other information related to a product area I was researching. The hardest part about this was understanding PubMed’s eutils and their .xml formats.

I did not want to manually enter the citation information. I wanted to avoid Zotero and other similar services because I wanted to integrate the data across domains and use the database that I was developing for them as the basis for a paper I wrote. For the PubMed citations, what I wanted to be able to do was to go to PubMed, perform a search, find an interesting citation, copy the PubMed URL, paste it into my Rails application, which would use the URL to query PubMed in XML format and populate my app’s database with that entry.

I have two models that play into this interaction: Article and Journal. I should note that I’m using Nokogiri to parse the XML and thoughtbot’s Paperclip gem to attach articles when they are available freely. This was done using Rails 2.3.5 and Ruby 1.8.7.

Here’s how the model files look:

journal.rb

class Journal < ActiveRecord::Base   attr_accessible :name, :description, :url, :short_name      has_many :articles      validates_uniqueness_of :name   validates_presence_of :short_name   validates_presence_of :name      before_save :make_short_name      def best_name     if !self.short_name.nil? && self.short_name.length > 0
      return self.short_name
    else
      return self.name
    end
  end

  private

  # Makes sure that a new journal has a short name in case one isn't entered.
  def make_short_name
    if self.short_name.nil? || self.short_name.length == 0
      self.short_name = self.name
    end
  end
end

The Article model is a little more complicated. The logic that I’m using the parse the XML isn’t great, but I’ve found that the structure changes from article to article. I’m still refining it and will update it when I’m finished, after I’ve found the 90% of article that auto-import.

article.rb

class Article < ActiveRecord::Base   require 'open-uri'   attr_accessible :pubmedlink,                    :article_title,                    :abstract,                    :authors,                    :affiliations,                    :pubdate,                    :pubmedid,                    :journal_id,                    :journal_volume,                    :journal_issue,                    :journal_pages,                   :fetched,                   :thearticle_file_name,                   :thearticle_content_type,                   :thearticle_file_size,                   :thearticle_updated_at,                   :thearticle                      has_attached_file :thearticle,                     :path => ":rails_root/public/system/uploads/:class/:id/:basename.:extension",
                    :url => "/system/uploads/:class/:id/:basename.:extension"
  belongs_to :journal
  before_save :make_pubmedid
  validates_uniqueness_of :article_title

  def citation
    cit = ""
    if self.journal_id != nil && self.journal_id > 0
      cit += self.journal.best_name + ". " + self.pubdate.year.to_s + " " + self.pubdate.strftime("%b")
    end
    if self.journal_volume != nil && self.journal_volume.length > 0
      cit += "; " + self.journal_volume
    end
    if self.journal_issue != nil && self.journal_issue.length > 0
      cit += "(" + self.journal_issue + ")"
    end
    if self.journal_pages != nil && self.journal_pages.length > 0
      cit += ": " + self.journal_pages
    end
      return cit
  end

  private

  def make_pubmedid
    if self.pubmedlink != nil && self.pubmedlink.length > 0 && self.fetched == false
      regex = Regexp.new(/[0-9]{4,10}/)
      matchdata = regex.match(self.pubmedlink)
      self.pubmedid = matchdata[0]
      pull_pubmed_data(matchdata[0])
      self.fetched = true
    end
  end

  def pull_pubmed_data(theID)
    @doc = Nokogiri::XML(open("http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=" + theID.to_s + "&retmode=xml").read)

    # The title and abstract
    self.article_title = @doc.xpath("//ArticleTitle").collect(&:text).to_s
    self.abstract = @doc.xpath("//AbstractText").collect(&:text).to_s

    # Setting up the authors
    first_names = @doc.xpath("//ForeName").collect(&:text)
    last_names = @doc.xpath("//LastName").collect(&:text)
    full_names = []

    i = 0
    last_names.length.times do
      the_name = first_names[i].to_s + " " + last_names[i].to_s
      full_names.push the_name
      i += 1
    end
    self.authors = full_names.join(", ")

    # Affiliations
    self.affiliations = @doc.xpath("//Affiliation").collect(&:text).to_s

    # Publication Date - Check if the complete date is at the top. If not use the pub med date.
    theyear = @doc.xpath("//PubDate/Year").collect(&:text)
    theyear = theyear[0]
    themonth = @doc.xpath("//PubDate/Month").collect(&:text)
    themonth = themonth[0]
    theday = @doc.xpath("//PubDate/Day").collect(&:text)

    if theyear.nil? || themonth.nil? || theday.nil? || theyear.length == 0 || themonth.length == 0 || theday.length == 0

      theyear = @doc.xpath("//PubMedPubDate[@PubStatus='pubmed']/Year").collect(&:text)
      theyear = theyear[0]

      themonth = @doc.xpath("//PubMedPubDate[@PubStatus='pubmed']/Month").collect(&:text)
      themonth = themonth[0].to_i
        month_done = 1
      if themonth < 10
        themonth = "0" + themonth.to_s
        month_done = 1
      end

      theday = @doc.xpath("//PubMedPubDate[@PubStatus='pubmed']/Day").collect(&:text)
    end

    if month_done != 1

      themonth = case themonth || "01"
        when "Jan"  then "01"
        when "Feb"  then "02"
        when "Mar"  then "03"
        when "Apr"  then "04"
        when "May"  then "05"
        when "Jun"  then "06"
        when "Jul"  then "07"
        when "Aug"  then "08"
        when "Sep"  then "09"
        when "Oct"  then "10"
        when "Nov"  then "11"
        when "Dec"  then "12"
      end

    end

    if theday.length == 0
      theday = "01"
    elsif theday[0].to_i < 10       theday = "0" + theday[0].to_s     else       theday = theday[0].to_s     end          thedate = theyear.to_s + "-" + themonth.to_s + "-" + theday.to_s     puts "thedate: " + thedate.to_s     self.pubdate = Date.new(theyear.to_i, themonth.to_i, theday.to_i)          # Either referencing the proper journal or creating a new one     thejournal = @doc.xpath("//Journal/Title").collect(&:text)     thejournal = thejournal[0]          theshortname = @doc.xpath("//MedlineTA").collect(&:text)     if theshortname.length == 0       theshortname = ""     else       theshortname = theshortname[0]     end          thejournalid = Journal.find(:first, :conditions => ['lower(name) = ?', thejournal.downcase])

    if !thejournalid.nil?
      self.journal_id = thejournalid.id
    else
      @journal = Journal.new(:name => thejournal.to_s, :short_name => theshortname.to_s)
      @journal.save
      thenewjournal = Journal.find(:first, :order => 'created_at DESC')
      self.journal_id = thenewjournal.id
    end

    # Save the volume, issue, and pages
    thevolume = @doc.xpath("//JournalIssue/Volume").collect(&:text)
    thevolume = thevolume[0].to_s if thevolume.length > 0
    self.journal_volume = thevolume if thevolume.length > 0

    theissue = @doc.xpath("//JournalIssue/Issue").collect(&:text)
    theissue = theissue[0].to_s if theissue.length > 0
    self.journal_issue = theissue if theissue.length > 0

    thepag = @doc.xpath("//Pagination/MedlinePgn").collect(&:text)
    thepag = thepag[0].to_s if thepag.length > 0
    self.journal_pages = thepag if thepag.length > 0

  end

end

The journals_controller.rb file is pretty standard. So is the articles_controller.rb file. Here it is as an example:

articles_controller.rb

class ArticlesController < ApplicationController   def index     @articles = Article.all(:order => "pubdate DESC")
  end

  def show
    @article = Article.find(params[:id])
  end

  def new
    @article = Article.new
    @journals = Journal.all(:order => :name)
  end

  def create
    @article = Article.new(params[:article])
    if @article.save
      flash[:notice] = "Successfully created article."
      redirect_to @article
    else
      render :action => 'new'
    end
  end

  def edit
    @article = Article.find(params[:id])
    @journals = Journal.all(:order => :name)
  end

  def update
    @article = Article.find(params[:id])
    if @article.update_attributes(params[:article])
      flash[:notice] = "Successfully updated article."
      redirect_to @article
    else
      render :action => 'edit'
    end
  end

  def destroy
    @article = Article.find(params[:id])
    @article.destroy
    flash[:notice] = "Successfully destroyed article."
    redirect_to articles_url
  end
end

Here are the view files for articles:

app/views/articles/index.html.erb

<div class="toolbar">
<ul></ul>
</div>
<table>
<tbody>
<tr>
<th></th>
<th>Article Title</th>
<th></th>
</tr>
<tr class="<%= cycle('rowA', 'rowB') %>">
<td></td>
<td><strong> </strong>

		  <em> </em></td>
<td class="deletelink"></td>
</tr>
</tbody>
</table>

app/views/articles/show.html.erb

<div class="toolbar">
<ul></ul>
</div>
	  <strong>PubMed Link:</strong>







	  <strong>View the Article:</strong>





  <strong>Article Title:</strong>



  <strong>Abstract:</strong>



  <strong>Authors:</strong>



  <strong>Affiliations:</strong>



  <strong>Pubdate:</strong>



  <strong>Pubmedid:</strong>



  <strong>Journal:</strong>



  <strong>Journal Volume:</strong>



  <strong>Journal Issue:</strong>



  <strong>Journal Pages:</strong>



app/views/articles/_form.html.erb



<h3>Enter a link to an abstract on PubMed and click Submit or fill out the information below</h3>











		There currently is no file attached.


<hr />












































The Paperclip gem requires :html => { :multipart => true } at the top of the _form file. It’s a very simple gem to use and very powerful. There are Google Groups for both Paperclip and Nokogiri, should you need additional assistance setting them up.

Leave me a comment if this helps you with either Nokogiri or with extracting data from PubMed!

Written by Clay

July 9, 2010 at 18:18

Posted in Code, Ruby, Technology

Tagged with , , , ,