Plugin: CSVGrab

I could add some extra debugging information to the plugin and email it you. This might give us a bit more to go on.

9/10 times the loading icon just spins in firefox and NOTHING happens

This suggests to me that something more serious is going wrong. Loading a 3 line CSV file and dumping the data into the database should be quick - it runs in no time on my install of EE and I’ve had reports on several thousand line imports happening in seconds.

Hey andrew apologies for the delay in getting back to you. you may have seen my other thread where my sites are now fubar’d because of a process eating far too much php memory.

Is there a way to check CSVGrab is no longer running? At the moment I’m turning off as much as i can and moving things around to try and find the culprit. not sure what it could be. not that up on php memory usage.

Hi David

I had noticed the other thread - hope you manager to sort it out soon.

not that up on php memory usage.

Me neither, sorry, but I’ll try and help.

The CSVGrab plugin - as it is called from a template - will only run when that page is loaded and will only run for the duration of time it takes that page to load. Once a page has loaded, or has been timed-out by the web server, the plugin will stop running (the plugin does not do anything to extend the page load timeout)

If that page is not getting loaded then the CSVGrab plugin should not be running in the background anymore.

Cheers Andrew. I think its safe to assume its not your plugin thats caused the problem. its something else. I think its something outside of EE which is annoying cos i dont use much outside of EE! Hopefully when i get this issue squared away and get CSVgrab working. your plugin looks like it could save me many many hours of work.

Hey Andrew. I’m back. And things are still not working.

After trying some new data and getting nothing but a spinning loading icon in Firefox and empty entries in EE I’m back begging for help.

If i understand everything correctly this should be very simple but I can’t get it to work.

My csv file looks like this

File Name    Capture Date    Category    Location    Creator
djw_20061117_28649.jpg    17/11/2006 19:16:10    Bird Lane    Cafe de Paris    David Webb
djw_20061117_28718.jpg    17/11/2006 20:03:17    Belgrave Scandal    Cafe de Paris    David Webb
djw_20061117_28760.jpg    17/11/2006 20:39:32    Scouting for Girls    Cafe de Paris    David Webb
djw_20061117_28836.jpg    17/11/2006 21:22:32    Educated Animals, The    Cafe de Paris    David Webb

In the future I want to add 3 more fields but for now I just want this in my db

File Name maps to the “title” field Category maps to the “v2_img_band” field and Location maps to the “v2_img_venue” field.

I’d like “Capture Date” to map to the “entry date” and “Creator” map to the “Author”. I can’t see how to do that (I’m hoping its possible but figured it wouldn’t hurt to have the info in the CSV and ignore it at the moment.)

Now my template has this in it

{exp:csvgrab url="http://acuityimages.com/20061177_me.csv"
    weblog="16"
    title="1"
    skip="1"
    delimiter="TAB"
    encloser=""
    use="3|4"
    fields="v2_img_band|v2_img_venue" }

I can add ” ” to the CSV if its recommended but I’m trying to keep things simple.

As I’m finishing writing this the template it still loading and I have one empty entry in my “images” weblog.

I can give you a login to my site if you think it’d help you troublehsoot but at the moment I’m at a loss as to what to do or try.

Thanks in advance for any and all help you can offer.

I can give you a login to my site if you think it’d help you troublehsoot

That might be helpful. For now, can you send me a copy (or link) to your CSV file. The one you have in your plugin was giving me a 404 error.

Cheers Andrew Just sent you a pm.

The correct url to the csv is http://acuityimages.com/20061117_me.csv I typed it wrong in the forum.

Hi Andrew,

I was just about to start writing a plug-in/module along these lines and then stumbled upon yours. I have quite a bit of additional functionality that is pretty specific to the project I’m working on though. I wonder how you would feel if I used this as a jumping off point for my own module for this project?

Basically we’re going to be using an uploaded CSV file to keep a database of member companies up to date on the website. In doing so there are a few things that need to happen:

• On an upload, if the company is new the system needs to generate a random password for that company and enter that into the appropriate field. That password needs to be emailed to an address that is provided in the CSV itself (the main company contact). • If the company is already in the system it needs to check and see if anything has changed - if it has then update those fields. • If the company is currently in the system but isn’t in the CSV it needs to delete that company from the database. • After upload we’re going to count the number of current member companies and generate an image based on that number (for the home page).

To start I’ll probably just have a page where someone can upload the file and everything happens automatically. Depending on how much time I have I should probably make an area in the CP where this happens and gives important feedback before the CSV is accepted. Which companies are being added, which are being removed, etc.

Any thoughts about the things above? Would you be ok if I use your plug-in as a jump-start on my own? Obviously any work I do I would give back to you if you’d like to use it…

Thanks Andrew.

I wonder how you would feel if I used this as a jumping off point for my own module for this project?

Please feel free to use this plugin as a jumping off point. Some of the code is a bit messy at the moment - a tidy-up is on my to do list. Also, on the to do list is the ability to do updates but you’ll probably get that working before I get a chance to…

Let me know if I can be any help

Thanks Andrew 😊

I’ll keep you posted - I’ll probably be working on this either next week or the week after. Just wanting to get all my ducks in a row.

Andrew - I ended up starting from scratch. I looked through your plug-in and I decided it would be just as easy as we’re coming at things from a slightly different angle - also it’s my first module so I thought I’d go through the tutorial and learn things from the ground up. I’ve got it up and running now - I’ve got a couple questions I’m going to post in the module tech support forum if you’re interested in looking at the module 😊

Just so I am clear what this plugin does… it technically only links to the .csv file and diplays it in a template. It doesn’t physically import it into the database.

Let me know if I am wrong.

I have a .csv file I want to permanetly import into a new weblog with custom fields. I am hoping this will do the trick.

The CSVGrab will import data into a weblog as you require.

Is there a detailed how to page I can referrence for this plugin. I am confused about the template code people are discussing and how I can import the data from one of my csv files into a new weblog.

Sorry for this newbee question.