Tuesday, January 26, 2010

Automating sftp with expect script

I got a request to automate the sftp process. I find it difficult with shell script. I find an easy way to do with
expect script.


The following packages need to be installed on the SUN server for the expect script. I downloaded the x86packages from sun freeware as I'm running the script from an x86 server.

tcl-8.5.3-sol10-x86-local
libgcc-3.4.6-sol10-x86-local
expect-5.43.0-sol10-x86-local


1)pkgadd -d tcl-8.5.3-sol10-x86-local

2)pkgadd -d libgcc-3.4.6-sol10-x86-local

The following packages are available:
  1  SMClgcc346     libgcc
                    (x86) 3.4.6
Installation of was successful.

3)pkgadd -d expect-5.43.0-sol10-x86-local 

 
The following packages are available:
  1  SMCexpect     expect
                   (x86) 5.43.0
Installation of was successful.


The expect will installed in the /usr/local/bin directory.

The script is as follows

----------------------------------------------------------------------------------------------------------
#!/usr/local/bin/expect -f
#This is the expect script wrote to automate the sftp process to pull files from a server as per the date and #then push the files to another server.
set timeout -1
set DATE [exec date "+%Y%m%d"]
log_file  "/home/ftp/logs/sftp.log"
send_log --  "####-Starting  SFTP script-  [exec date] \n"
send_log --  "Today's date:[exec date] \n"
send_log --  "Downloading files through sftp \n"
spawn /usr/bin/sftp jim@172.20.1.86
expect "Password:"
#sleep 5
send "jim123\n"
expect "sftp>"
send "cd /export/home/jim \r"
expect "sftp>"
send "ls \r"
expect "sftp>"
send "get *.$DATE \r"
expect "sftp>"
send "quit \r"
sleep 5
send_log --  "Today's date:[exec date] \n"
send_log --  "Uploading files through sftp \n"
spawn /usr/bin/sftp jim@172.20.5.93
expect "Password:"
send "jim123\n"
expect "sftp>"
send "cd /export/home/jim \r"
expect "sftp>"
send "put *.$DATE \r"
expect "sftp>"
send "quit \r"
#END of SCRIPT
-----------------------------------------------------------------------------------------------------------

I can shell script from expect script. I have another requirement to pull yesterdays file through sftp server. Thank god that was a Linux server and to get yesterdays date in Linux is easy :)

set DATE [exec /home/ftp/date.sh ]

The above line will call the date.sh script to get yesterdays date. 
date.sh script is as below

[root@]# cat date.sh
#!/bin/bash
echo $(date  --date='1 day ago' "+%Y%m%d")
#End of script









11 comments:

  1. Hi Jibby George,

    Thanks a lot for posting this.
    I was really fed up googling this, but finally stick to your site and works perfectly.

    Thankyou Man. you are so special. you made it so easy.

    ThankYou

    ReplyDelete
  2. Hi Jibby,
    I have a similar requirement. Your method of doing that is impressive. I tried to follow this. But, the glitch here is that the sftp user name has a special character $ in its name. While running the script the spawn/expect/send libraries are treating the text after $ symbol as a variable. Is there a way that you think of to get away with that. I tried single quotes, double quotes around the user name but no luck.

    Thanks for your help.

    Naresh

    ReplyDelete
  3. HI Naresh , thanks for that. Its been a while I worked with expect. There should be a way to do that. If I figure out something will post here.

    Jibby

    ReplyDelete
  4. Great! Thanks much for your help Jibby.

    Naresh

    ReplyDelete
  5. Hi Jibby, found a solution to get away with the $ symbol using expect. When I place the \ (backward slash) in front of the $ symbol, then it works fine.

    Thanks!
    Naresh

    ReplyDelete
  6. How can I check to see if the password is invalid or expired? I have a similar task to automate. Nice script.

    ReplyDelete
  7. I was able to figure out my check for invalid or expired password. The system I was connecting to does not tell you if the password is invalid, it just gives the password prompt over and over that is why there is a second expect for the password prompt. I also put in a check for upper and lower case prompts. After doing the spawn of the sftp command it goes like this:

    spawn sftp mylogonid@someserver
    expect -re "password|Password:"
    send "mypasswd\r"
    expect -re "password|Password:" {
    puts "Password was not accepted \n"
    exit 2
    } -re "expired|Expired" {
    puts "Password Expired \n"
    exit 3
    } -re "invalid|Invalid" {
    puts "Password Invalid \n"
    exit 4
    } "sftp>" {
    send "pwd\r"
    expect "sftp>"
    send "put myfilename yourfilename\r"
    expect "sftp>"
    send "bye\r"
    }

    ReplyDelete
    Replies
    1. I have a simple requirement and try to figure out how to do it in EXPECT.

      after sftp into a server. I want to cd into the latest directory (Note $HOME has directories and files). Any idea a easy way to achieve this??

      Thanks

      Delete
  8. Worked great! Thank you!

    ReplyDelete
  9. Hi

    I have problem with file transmission (file 200MB +/- 50MB).
    The trasmission had been interrupted before finished dowloading. Rest works good. Any ideas ??

    ReplyDelete