Plugin: FeedGrab

OK, quick question. We have 3 weblogs: A, B, and C. We are using FeedGrab to cross-post new weblog entries on the other weblogs within our same EE install. If I want to use FeedGrab to pull in new content from A onto B and C, how can I tell that the posts are unique? Right now, I can only grab unique content on ONE of our sites. Is there a value of “weblog” that can be put into the unique=”” parameter? This would allow me to make sure that posts have not been posted more than once on the SAME weblog, but would allow me to post the same thing on B and C if required. Any ideas?

Thanks,

Doug

I’m trying to use FeedGrab .7.6 for the first time, and I get this error:

Fatal error: Call to undefined function: mb_convert_encoding() in /home/…/plugins/pi.feedgrab.php on line 185

I’m running PHP Version 4.3.11 and EE 1.6.3.

Ideas?

TTFN Travis

If I want to use FeedGrab to pull in new content from A onto B and C, how can I tell that the posts are unique? Right now, I can only grab unique content on ONE of our sites.

Hi Doug,

The plugin should actually do what you are trying to do. I’ll try and get a fix for you this week.

Andrew

Fatal error: Call to undefined function: mb_convert_encoding() in /home/…/plugins/pi.feedgrab.php on line 185

The plugin is trying to use a function without checking whether you have the correct php module installed.

I’ll try and get a fix for this to you this week.

Andrew

I love this plugin, it enables me to do what I’ve been looking for for a long time! It works perfectly for most of my feeds. Just 2 small problems:

• When I try to get my delicious tags as categories, only the first tag for each entry is imported. I copied the code from page 3 of this thread, changed only the feed address, the category group_id and the fields. Has something changed since then? I read the whole thread and the documentation, but could not find an answer.

• Fetching a blogger feed does not import the content field, the field has the text “array” instead.

Any hints? I haven’t looked very closely at RSS coding yet, so maybe I’m just missing something obvious.

(eta) One more question, is there a way to import flickr tags as categories? The rss source code looks like this:

<category term="tagnr1" scheme="http://www.flickr.com/photos/tags/" >
        <category term="tagnr2" scheme="http://www.flickr.com/photos/tags/" >
        <category term="tagnr3" scheme="http://www.flickr.com/photos/tags/" >

Whatever I try, I cannot get the tags to translate into categories…

Hey Andrew -

Great work on this! Another (very small) thing that would be handy to add is a function that increases the post count for the author once the post is pulled over. The ‘exp_members’ table has a ‘total_entries’ field that is not getting updated. This is actually making it so I can’t use the “Filter by Author” plugin for that member. Not a huge deal, but I figured it can’t hurt to get this request out there.

Thanks!

Doug

Hi Andrew. Any luck on that update?

TTFN Travis

I modified our copy of the plugin to fix the charset error Travis mentions above (appx line 186):

if (function_exists('mb_convert_encoding'))
{
  $encoded_title = mb_convert_encoding($this->post[ "title" ], strtoupper('UTF-8'), $source_encoding );
}
elseif(function_exists('iconv') AND ($iconvstr = @iconv($source_encoding, 'UTF-8', $this->post["title"])) !== FALSE)
{
  $encoded_title = $iconvstr;
}
else
{
  $encoded_title = utf8_encode($this->post["title"]);
}

The logic is from core.regex.php.

Justin

I hate to sound like an idiot here but I’m not getting anything from this plugin. The plugin is installed. I followed the tutorial: created a new weblog, assigned the custom field groups in accordance to the XML nodes of the RSS feed. Created a new template group with an index file and a go file. Double checked that the weblog ID is correct. The go template just displays:

Fetching: http://www.joomlashack.com/index.php?option=com_rd_rss&id=2&Itemid=33

nothing else..

I’m using EE v 1.6.4 with FeedGrab (v.0.7.6)

Is this supposed to work with the above versions?

EDIT: After enabling debug, I see now that my error is also related to the mb_convert

I removed:

// Get unique url
            $source_encoding = ( isset( $this->RSS->source_encoding ) ? $this->RSS->source_encoding : 'UTF-8' );
            $encoded_title = mb_convert_encoding($this->post[ "title" ], strtoupper('UTF-8'), $source_encoding ); // Fixes warning on next line
            $this->post[ "url_title" ] = $REGX->create_url_title( $encoded_title );
            $sql = "SELECT count(*) AS count FROM exp_weblog_titles WHERE url_title = '".$DB->escape_str( $this->post[ "url_title" ] )."' AND weblog_id = '$this->feed_weblog_id'";
            $results = $DB->query($sql);
            if ($results->row['count'] > 0) {
                // Duplicate exists, add a counter on the end
                $sql = "SELECT count(*) AS count FROM exp_weblog_titles WHERE url_title LIKE '".$DB->escape_str( $this->post[ "url_title" ] )."%' AND weblog_id = '$this->feed_weblog_id'";
                $results = $DB->query($sql);
                $this->post[ "url_title" ] .= $results->row['count']+1;
            }

to suppress the errors and it imported fine..

OK - tried this for the first time importing my Tweets into my personal EE blog space.

First time it choked because of some weirdness I had with the new weblog/new custom fields. I deleted both and started over from scratch.

Second time - worked great! Imported in the 20 posts.

Although I just discovered EE’s 100 char. limit in the Title field. Doh! This results in many tweets being truncated (especially with the inclusion of the username, see below). Probably best to add another field for the tweet itself for anyone trying to do this too.

Two things I need to figure out now:

Twitter’s pubDate is not adjusted for my timezone, i.e. the date saved with my entry is 7 hours ahead of what it should be. Is there a way to modify this at the time of grabbing the RSS entry?

Secondly, the problem with the username affixed to the beginning of the title. I found this Trimmer plugin that can do it when I output the data, but again is there a way to parse this during the time of the RSS grab?

And I’m with klick above - need to be able to access that lower level data in FriendFeed AND be able to extract the two different titles from Google Reader shared items which currently look like this:

<link rel="alternate" href="http://www.kottke.org/remainder/08/06/15936.html" type="text/html"/>
<link rel="related" href="http://www.kottke.org/" title="kottke.org"/>

Obviously the “alternate” one would be the one I’d like to use as the Title. “related” would be good for a “via” link (like the Google Reader Shared page).

There is an offset function in FeedGrab that allows you to modify date; but the one person I know who used it, was getting duplicate entries. Though admittedly, they were making other modifications as well. So I’d give it a try and see how it works for you. Make sure you have the most current version. I don’t use FriendFeed, so no comments on best method for that.

Hopefully there’s still work being done on this because I’ve got lots of uses for it in the coming months!

Added the offset and it works great (value is in seconds, for West Coasters that’s -25,200). I can see there being a problem when DST switches. Probably not worth the effort to incorporate that into plugin (easy change twice a year).

I just tried running it again with the unique value set and it worked great - no duplicate entries, I did get this error code though:

Notice: Undefined property: MagpieRSS::$last_modified in .../plugins/pi.feedgrab.php on line 1711

Hope this helps.

REMOVED - not pertinent to this thread

OK - trying to break down the FriendFeed RSS using FeedGrab.

Like klick above I can get title, pubDate, and the link. But what I am really interested in is grabbing my comment within that entry. With the RSS feed, my comments feed is formatted like this:

<ff:comments>
  <ff:comment>
    <ff:date>2008-07-08T18:39:53Z</ff:date>
    −<ff:body>
       This is where my comment is!
     </ff:body>
    −<ff:user>
       <ff:id>This is where my USER ID is</ff:id>
       <ff:name>ny name</ff:name>
       <ff:nickname>myname</ff:nickname>
       <ff:profileUrl>http://friendfeed.com/myname</ff:profileUrl>
     </ff:user>
   </ff:comment>
</ff:comments>

The problem is this might be within a list of 30-40 comments depending on the entry. Is there someway, using this plugin or XMLGrab, where I can check my USER ID, and if it is me, put the <body> of that content into my weblog? Can I use the @ to access this info?