FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora User

 
 
LinkBack Thread Tools
 
Old 10-13-2012, 01:50 AM
JD
 
Default wget or curl and yousendit.com

Is there a way to use wget or curl or clive to download files from
yousendit.com?

Would appreciate some examples.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-13-2012, 05:47 AM
Mark LaPierre
 
Default wget or curl and yousendit.com

On 10/12/2012 09:50 PM, JD wrote:

Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.


It looks like you have to log in to yousendit to get access to your
file. That would indicate that it is not scriptable with anything like
wget. You would have to get the login page and then provide the
required credentials. There is almost always a Captcha that you have to
read and decode put there just to prevent the kind of thing you want to do.


--
_
v
/(_)
^ ^ Mark LaPierre
Registerd Linux user No #267004
https://linuxcounter.net/
****
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-13-2012, 10:08 PM
JD
 
Default wget or curl and yousendit.com

On 10/12/2012 11:47 PM, Mark LaPierre wrote:

On 10/12/2012 09:50 PM, JD wrote:

Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.


It looks like you have to log in to yousendit to get access to your
file. That would indicate that it is not scriptable with anything
like wget. You would have to get the login page and then provide the
required credentials. There is almost always a Captcha that you have
to read and decode put there just to prevent the kind of thing you
want to do.



Actually, no. I am not trying to access my own files that are UPLOADED
to yousendit.com
I am trying to download files that another user uplpoaded and made available
to a distribution list.
Usually, I have to click on each link and download it - no login required.
But in some of the email notifications I get from yousendit.com, the
message contains many files to download, and I am trying to avoid having
to click on each and every one, to download them.

Each file link, when you cright-click it, takes you to the full link of
the file,

and you can then use that link as an arg to wget.



--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-14-2012, 12:12 AM
Ian Malone
 
Default wget or curl and yousendit.com

On 13 October 2012 23:08, JD <jd1008@gmail.com> wrote:
>

>>>
>>> Is there a way to use wget or curl or clive to download files from
>>> yousendit.com?
>>> Would appreciate some examples.

> Each file link, when you cright-click it, takes you to the full link of the
> file,
> and you can then use that link as an arg to wget.
>
>
>
>

If feeling brave you could script link extraction from the page and a
second curl request, alternatively wget's recursive -r option may be
what you want:
http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html

--
imalone
http://ibmalone.blogspot.co.uk
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-14-2012, 12:26 AM
Mark LaPierre
 
Default wget or curl and yousendit.com

On 10/13/2012 08:12 PM, Ian Malone wrote:

On 13 October 2012 23:08, JD<jd1008@gmail.com> wrote:






Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.



Each file link, when you cright-click it, takes you to the full link of the
file,
and you can then use that link as an arg to wget.






If feeling brave you could script link extraction from the page and a
second curl request, alternatively wget's recursive -r option may be
what you want:
http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html



Here's a bit of a guide to build your own script.

# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE"


sleep 1

# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc"
"rundown.php?prgId=2" | sort | uniq | awk -F""" '{print $4}' | awk
-F"?" '{print $1}'`

do
wget $URL 2> /dev/null &
sleep 1
done


--
_
v
/(_)
^ ^ Mark LaPierre
Registerd Linux user No #267004
https://linuxcounter.net/
****
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-14-2012, 02:14 AM
JD
 
Default wget or curl and yousendit.com

On 10/13/2012 06:26 PM, Mark LaPierre wrote:

On 10/13/2012 08:12 PM, Ian Malone wrote:

On 13 October 2012 23:08, JD<jd1008@gmail.com> wrote:






Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.


Each file link, when you cright-click it, takes you to the full link
of the

file,
and you can then use that link as an arg to wget.






If feeling brave you could script link extraction from the page and a
second curl request, alternatively wget's recursive -r option may be
what you want:
http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html





Here's a bit of a guide to build your own script.

# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE"


sleep 1

# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc"
"rundown.php?prgId=2" | sort | uniq | awk -F""" '{print $4}' | awk
-F"?" '{print $1}'`

do
wget $URL 2> /dev/null &
sleep 1
done



Will try it. Thanks for the example.

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-14-2012, 04:03 AM
JD
 
Default wget or curl and yousendit.com

On 10/13/2012 06:26 PM, Mark LaPierre wrote:


Here's a bit of a guide to build your own script.

# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE"


sleep 1

# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc"
"rundown.php?prgId=2" | sort | uniq | awk -F""" '{print $4}' | awk
-F"?" '{print $1}'`

do
wget $URL 2> /dev/null &
sleep 1
done

Thank you Mark.

After looking into the source page containing all the links, and saw how the
URL of each file to download is constructed, I built and ran the
following script

which worked like a charm.
PS: The first URL given to wget came to me in an email message sent to me by
yousendit.com because the uploader has me in his distribution list on
yousendit.com.


#!/bin/bash
wget -c -O ysi.txt
'https://www.yousendit.com/dl?phi_action=app/orchestrateDownload&rurl=https........etc'
grep 'directDownload?' ysi.txt | sed -e
's/.*directDownload?/https://www.yousendit.com/directDownload?/' -e
's/=bas.*$/=bas/' | sort | uniq > urls # so that I can examine this
file in case my sed script has an error

while read url; do
wget -c -ndH "$url"
done < urls

Thanx for the inspiration Mark.



--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 10-14-2012, 05:11 AM
Mark LaPierre
 
Default wget or curl and yousendit.com

On 10/14/2012 12:03 AM, JD wrote:


On 10/13/2012 06:26 PM, Mark LaPierre wrote:


Here's a bit of a guide to build your own script.

# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE"

sleep 1

# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc"
"rundown.php?prgId=2" | sort | uniq | awk -F""" '{print $4}' | awk
-F"?" '{print $1}'`
do
wget $URL 2> /dev/null &
sleep 1
done

Thank you Mark.

After looking into the source page containing all the links, and saw how
the
URL of each file to download is constructed, I built and ran the
following script
which worked like a charm.
PS: The first URL given to wget came to me in an email message sent to
me by
yousendit.com because the uploader has me in his distribution list on
yousendit.com.

#!/bin/bash
wget -c -O ysi.txt
'https://www.yousendit.com/dl?phi_action=app/orchestrateDownload&rurl=https........etc'

grep 'directDownload?' ysi.txt | sed -e
's/.*directDownload?/https://www.yousendit.com/directDownload?/' -e
's/=bas.*$/=bas/' | sort | uniq > urls # so that I can examine this file
in case my sed script has an error
while read url; do
wget -c -ndH "$url"
done < urls

Thanx for the inspiration Mark.





Great minds think alike. ;-)

--
_
v
/(_)
^ ^ Mark LaPierre
Registerd Linux user No #267004
https://linuxcounter.net/
****
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 

Thread Tools




All times are GMT. The time now is 09:15 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org