If you run Robocopy with an RCJ file (Robocopy Job) just once, then the file is just that: a job file. However, if you plan to use the same settings over and over again, then consider this a configuration file that is easily modifiable and copyable to reuse on other directories.
I personally have a directory structure set up like:
Backups
|--BackUpJobs
| |--*.RCJ files
|--BackUpLogs
| |--*.lob files
|--*.bat files (for single jobs)
|--RunAllBUs.bat (to execute all jobs)
The log files should be self-explanatory. Here I just want to run through the bat and RCJ files.
For my individual bat files, I will use something like this for all single jobs:
@echo off
cd c:\users\myusername\desktop\BackUp
robocopy /JOB:BackUpJobs\[WhatItIs]BUJOB.RCJ
pause
Where [WhatItIs] is something indicating what directory I am backing up. For example, if I was backing up the CIS577 directory, the path would be: BackUpJobs\CIS577BUJOB.RCJ
As with any bat script, the path is relative to where the script is executing from.
Now for the fun part, the rcj "config" files. These files, if you know the syntax for robocopy, can be modified in short time to create a full backup system with a variety of operations. For instance, one job can do a /MOV, which will delete everything from the source after it's copied to the destination while another job just makes a copy of the directory and all subs(/E), copying new(er) files to the destination.
CIS577BUJOB.RCJ
::
:: Robocopy Job
::C:\USERS\MYUSERNAME\DESKTOP\BACKUP\BACKUPJOBS\CIS577BUJOB.RCJ
::
:: Created by myusername on Sun Apr 10 2011 at 20:46:13
:: Modified by hand. 15 May at 1210 am.
::
:: Source Directory :
::
/SD:C:\Users\myusername\Desktop\CIS577\ :: Source Directory.
::
:: Destination Directory :
::
/DD:\\werdenshare\GoFlex Home Personal\DaveSchoolMain\UofM_Dearborn\CIS577\
:: Destination Directory.
::
:: Include These Files :
::
/IF :: Include Files matching these names
:: *.* :: Include all names (currently - Command Line may override)
::
:: Exclude These Directories :
::
/XD :: eXclude Directories matching these names
:: :: eXclude no names (currently - Command Line may override)
::
:: Exclude These Files :
::
/XF :: eXclude Files matching these names
:: :: eXclude no names (currently - Command Line may override)
::
:: Copy options :
::
/S ::Copy Subdirs but not empty ones
/E ::Copy Subdirs including empty ones
/COPY:DAT :: what to COPY (default is /COPY:DAT).
::
:: Retry Options :
::
/R:1000000
:: number of Retries on failed copies: default 1 million.
/W:30
:: Wait time between retries: default is 30 seconds.
::
:: Logging Options :
::
/LOG+:C:\Users\myusername\Desktop\BackUp\BackUpLogs\CIS577BULog.log
:: output status to LOG file (overwrite existing log).
The RCJ file does nothing more than pass the parameters on the command line that you would be using if you didn't use the job file. So without this file, your robocopy job using the above would be:
$>robocopy C:\Users\myusername\Desktop\CIS577\ "\\werdenshare\GoFlex Home Personal\DaveSchoolMain\UofM_Dearborn\CIS577\"
/S /E /LOG+:C:\Users\myusername\Desktop\BackUp\BackUpLogs\CIS577BULog.log
A couple things to notice: the source and destination directories ONLY need to be wrapped in quotation marks IF either one has spaces AND is passed on the command line. In the RCJ file, no quotation marks needed. Also, if you notice the command line example parameters, you will see [source] [destination] /S /E /LOG+. and not the other options such as /XD from the file. This is becuase when a job you created is saved to an RCJ file, all defaults are written to the file unless you have passed a paremeter to overwrite their usage completely.
The really easy part that I like is that I can copy this file out, adjust the source and destination, at a minimum, and then save the file as another robocopy job file. The extra bit of ease here, if you know where the options go, is that you can easily add any option changes to the files as you create them or modify your needs. For example, in the Copy options section, I can add /MOV to the list of uncommented parameters and this will do what you'd expect as I mentioned before (although the folder/subfolder structure will remain intact.)
This is probably enough from me on Robocopy this year. :-) Now I am working on a perl script to take an exported list of IE and FireFox bookmarks and to create an XML file for these. Other than the easy answer of just wanting a quicker way to access good used references, the format I am going with (as created by my buddy James) will allow me to add usernames, masked passwords (if I am feeling crazy), and/or password hints. Additionally, I am going to take it a step farther for another display field for things such as Frequent Flyer program numer and POC info. Really this is an academic exercise to create something I want...I get a little tired of scrolling through a TON of bookmarks on a LOT of different computers. By doing this, I can keep it updated and portable....basically a poor-man's way to sync some favorites between computers.
Showing posts with label batch files. Show all posts
Showing posts with label batch files. Show all posts
Wednesday, August 31, 2011
Tuesday, February 23, 2010
Combining the pcap files quickly
<>
An updated to this post, based on the last comment that I had recieved:
I agree that Pcapjoiner and some other tools can do this quickly, as well as add some other functions.
I like the fact though that there are these tools built into Wireshark that allows for the quick combo of just a few targeted pcaps. Basically, a way to get to the down and dirty of analysis on one more connections.
This has given me a GREAT idea for two/three/four more blog posts that would love to do:
1) building my own interface to do these merges, and other massaging that might be helpful. I am already picturing a ton of ways to go with this...maybe best to keep it simple...but it could be a fun projec for myself...adding the ability to run som stats, filters, create some xml/xhtml/html output in addition to output usalbe by tcpdump, wireshark, and ngrep.
2) to play around with the options of mergecap from the command line and try to add some filters by piping to/from ngrep or tcpdump. I think this should work just fine, and would allow for a larger number of files to be processed easily by mergecap in the dump.
3) a perl script that I can just drag a group of files onto for the merge. Perl's CPAN modules provide some excellent support for network traffic
4) a perl script to strip out whatever I want from a fully captured session: the webpage, a pic, the VoIP call, etc. This one might be a little harder....but sounds REALLY fun to me.
dw
<>
The other day I finally became fed up with the process of using the Wireshark GUI on Windows to combine more than two PCAP files. I think some folks I know would give me a "Gibbs' Slap" if they knew just how many times I used the GUI to combine 15+ captures. (If you don't know what a "Gibbs' slap" is, you REALLY need to start watching the original NCIS, and NO, not the lousy NCIS:Los Angeles)
Unless things have changed (and I admit to not recently trying), it is generally easy in *nix to pass/search a directory of *.pcap files to the Mergecap.exe util of the Wireshark release, combining ALL the PCAP files into a specified output file. However, and I know this is [NOT] a shock to most people, it is not always as easy to do this same thing on the windows command line (which I was stuck using for this). Of course, I have 20-30 more years before I am a cmd line ninja, so there may be a very easy way to do this...but I don't know it and my friend Google couldn't find it. This left me with a huge whole in my life as I REALLY wanted a better and FASTER way to do this.
Before beginning this walk down my pcap-crazy mental train track, just a quick recap of how to use mergecap.exe:
Usage: mergecap [options] -w...
So if I just want to merge some pcap files from a desktop folder into a file called merged.pcap:
C:\Program Files\Wireshark\mergecap -w merged.pcap "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\1.pcap" "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\2.pcap" "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\3.pcap"
This will combine the 1.pcap, 2.pcap, and 3.pcap files into the newly created merged.pcap. However, in case it went un-noticed, that is a LOT of typing to combine three files. Isn't there an easier way?
The Choices:
1) Write a GUI that let me quickly select multiple files, creates the command line string for mergecap with these files, and executes the command. Great! Except, do I really want to create a GUI to do this?
2) Write a command line program to parse a specified folder for all pcaps, create the command string, then execute.
3) Create a script or bat file to do what I want, when I want.
4) Give up and begin a life of cheap booze and cheaper women.
The answers:
1) Nope. Little bit to lazy to spend the time to create the GUI that will make me spend more time navigating directory structures and selecting n files.
2) Nope. Lazy...see number 1 above.
3) This sounds like the way to go.
4) Might work, but then wife and kids might become irritated with such a choice. Back to number 3.
So now that it is decided that I am lazy, and can't chase women or whiskey, it's on to the scripting. There are multiple options here as well, but I kept it simple, dug up some examples, tweaked them for me, and went back to watching Office Space.
The batch file:
combine.bat:
setlocal
set myfiles=
for %%f in (*.pcap) do set myfiles=!myfiles! %%f
Cmd /V:on /c "c:\Program Files\Wireshark\mergecap.exe" -w temp.pcap %myfiles%
What does this mean and where does it go? I created a folder on my desktop for the pcaps I want to merge; the bat file goes here. To run this, I could double-click the file, but I prefer to see it in action . With that in mind, I open up a command prompt in the folder where the bat file is stored, and then execute:
$>combine.bat
The important things I want to point out here is the "Cmd /V:on /c ...". What this does:
- CMD /V:on re-calls the cmd.exe from the system32 directory with the setting of the delaying environment variables (/V:on).
- /c means to "run the following command." In this situation, no environment variables for the PATH for mergecap exist, so I need to call it directly, passing the remainder of the string as the arguments.
- myFiles is an array of all .pcap files in the directory where the script resides. Without the "/V:on" option, only the last file name passed by the 'for' will be present when the command executes.
This entry took longer to create then the batch file, but I hope it helps some angry analyst somewhere.
An updated to this post, based on the last comment that I had recieved:
I agree that Pcapjoiner and some other tools can do this quickly, as well as add some other functions.
I like the fact though that there are these tools built into Wireshark that allows for the quick combo of just a few targeted pcaps. Basically, a way to get to the down and dirty of analysis on one more connections.
This has given me a GREAT idea for two/three/four more blog posts that would love to do:
1) building my own interface to do these merges, and other massaging that might be helpful. I am already picturing a ton of ways to go with this...maybe best to keep it simple...but it could be a fun projec for myself...adding the ability to run som stats, filters, create some xml/xhtml/html output in addition to output usalbe by tcpdump, wireshark, and ngrep.
2) to play around with the options of mergecap from the command line and try to add some filters by piping to/from ngrep or tcpdump. I think this should work just fine, and would allow for a larger number of files to be processed easily by mergecap in the dump.
3) a perl script that I can just drag a group of files onto for the merge. Perl's CPAN modules provide some excellent support for network traffic
4) a perl script to strip out whatever I want from a fully captured session: the webpage, a pic, the VoIP call, etc. This one might be a little harder....but sounds REALLY fun to me.
dw
<
The other day I finally became fed up with the process of using the Wireshark GUI on Windows to combine more than two PCAP files. I think some folks I know would give me a "Gibbs' Slap" if they knew just how many times I used the GUI to combine 15+ captures. (If you don't know what a "Gibbs' slap" is, you REALLY need to start watching the original NCIS, and NO, not the lousy NCIS:Los Angeles)
Unless things have changed (and I admit to not recently trying), it is generally easy in *nix to pass/search a directory of *.pcap files to the Mergecap.exe util of the Wireshark release, combining ALL the PCAP files into a specified output file. However, and I know this is [NOT] a shock to most people, it is not always as easy to do this same thing on the windows command line (which I was stuck using for this). Of course, I have 20-30 more years before I am a cmd line ninja, so there may be a very easy way to do this...but I don't know it and my friend Google couldn't find it. This left me with a huge whole in my life as I REALLY wanted a better and FASTER way to do this.
Before beginning this walk down my pcap-crazy mental train track, just a quick recap of how to use mergecap.exe:
Usage: mergecap [options] -w
So if I just want to merge some pcap files from a desktop folder into a file called merged.pcap:
C:\Program Files\Wireshark\mergecap -w merged.pcap "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\1.pcap" "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\2.pcap" "c:\users\UnixUsersAreCooler\Desktop\Some Pcap Files\3.pcap"
This will combine the 1.pcap, 2.pcap, and 3.pcap files into the newly created merged.pcap. However, in case it went un-noticed, that is a LOT of typing to combine three files. Isn't there an easier way?
The Choices:
1) Write a GUI that let me quickly select multiple files, creates the command line string for mergecap with these files, and executes the command. Great! Except, do I really want to create a GUI to do this?
2) Write a command line program to parse a specified folder for all pcaps, create the command string, then execute.
3) Create a script or bat file to do what I want, when I want.
4) Give up and begin a life of cheap booze and cheaper women.
The answers:
1) Nope. Little bit to lazy to spend the time to create the GUI that will make me spend more time navigating directory structures and selecting n files.
2) Nope. Lazy...see number 1 above.
3) This sounds like the way to go.
4) Might work, but then wife and kids might become irritated with such a choice. Back to number 3.
So now that it is decided that I am lazy, and can't chase women or whiskey, it's on to the scripting. There are multiple options here as well, but I kept it simple, dug up some examples, tweaked them for me, and went back to watching Office Space.
The batch file:
combine.bat:
setlocal
set myfiles=
for %%f in (*.pcap) do set myfiles=!myfiles! %%f
Cmd /V:on /c "c:\Program Files\Wireshark\mergecap.exe" -w temp.pcap %myfiles%
What does this mean and where does it go? I created a folder on my desktop for the pcaps I want to merge; the bat file goes here. To run this, I could double-click the file, but I prefer to see it in action . With that in mind, I open up a command prompt in the folder where the bat file is stored, and then execute:
$>combine.bat
The important things I want to point out here is the "Cmd /V:on /c ...". What this does:
- CMD /V:on re-calls the cmd.exe from the system32 directory with the setting of the delaying environment variables (/V:on).
- /c means to "run the following command." In this situation, no environment variables for the PATH for mergecap exist, so I need to call it directly, passing the remainder of the string as the arguments.
- myFiles is an array of all .pcap files in the directory where the script resides. Without the "/V:on" option, only the last file name passed by the 'for' will be present when the command executes.
This entry took longer to create then the batch file, but I hope it helps some angry analyst somewhere.
Subscribe to:
Posts (Atom)