Каскадный · [ Стандартный ] · Линейный+ "A Guide to Pubbing" - mozhet komu prigoditsya

A Brief Introduction to FTP

The Protocol for electronic File Transfer (FTP) was first proposed in April 1971 by an M.I.T. researcher named Abhay Bhushan. FTP was founded by the U.S. military and scientific community`s research facilities, collectively known as 'DARPA' or Defence Advanced Research Projects Agency.

There was a wide mix of mainframe computers in place within DARPA at that time (VAX, VMS, Unix varieties) and so they needed a platform-independent interface. FTP was described as 'the user interface to the DARPA File Transfer Protocol' and 'a protocol for file transfer between HOSTs on the ARPANET' (rfc0959). In other words, FTP was the world`s first cross-platform server 'browsing' protocol.

The original intent for FTP was to allow the U.S. military to continue communicating in the event of a nuclear war, but otherwise ended up being used to facilitate the sharing of knowledge as well as to avoid duplication in scientific research. If you wanted to design some military device, for example, you could tap into the DARPA network and run a search on words and terms to be sure someone else within DARPA hadnt already done the same research and if they had, whether their prior discoveries might assist you in expediting your own research.

Initially developed at M.I.T, File Transfer Protocol was first implemented by the University of California, Berkeley for their variety of Unix (Berkeley Unix or BSD) in version 4.2BSD. During the late 1970s to mid 1980s the File Transfer Protocol was regularly updated with extensions and enhancements by which time the 'net' consisted of 562 servers, or HOSTS.

Several public FTP search engines, namely 'Gopher', 'Archie', and 'Veronica', were subsequently created to assist the public in tapping into the resources then available on the early Internet FTP sites. As well, the DARPA system included 'electronic correspondence' which later was spun-off, with 'e-mail' becoming the first killer-app of the early public Internet.

Finally, the people in charge at DARPA chose at some point to define stored data as being in the form of what was termed a 'page' of information which gave birth to the second 'killer-app' of the early Internet, the graphical page browser. The "Mosiac" browser was designed to allow the user to click a line of linked text, bringing that page onto the user`s screen, i.e. hyper-linked text. Later work on graphical browsers, namely Netscape Navigator, eventually led to the establishment of the modern World Wide Web.

In order to tap into the public FTP community that presently exists online, you need only locate any one of a number of Windows-based FTP apps on the market and then read its built-in help files. After you`ve done that, you should be able to make use of the information contained below.

------- ------- -------

*Setting Up Your FTP App*

In SmartFTP go to Tools/Settings/General to set it to anonymous user and enter any email addy as your password. In Tools/Settings/Advanced, make sure that Show Hidden is checked. SmartFTP cant see every type of hidden dir but it can see more of them than can FlashFXP which is why SmartFTP is recommended for use in checking a new server for signs of prior 'occupants', checking your pubs for signs of squatters, etc.

In FlashFXP, go to Options/Preferences and anonymous user is on the General tab. While at it, go to Options/Preferences/Advanced and make sure Show Hidden is checked. As with SmartFTP, most of the other default settings are okay as is for now.

If you are using BitBeamer, the options are in File/Preferences. Set any email as your password, and check Show Hidden.

*Logging onto a Remote Server*

To establish a connection with a remote server, you use a 'client', an FTP application, and simply type in the remote server`s IP (FTP is based on the TCP/IP Protocol). "The FTP protocol requires the FTP server to use two different ports, and manage two TCP connections. One is called the control connection over which all control information, such as user ID and password, is transmitted. All FTP commands and replies are also exchanged via this connection. For the control connection, the well-known port 21 is used as default. The other one is called the data connection which is used for transferring the contents of files based on the FTP client's requests." D. J. Bernstein.

*Communicating With the Server*

Log onto your chosen server and the first thing you`ll notice is that the remote server communicates with your FTP 'client' app by way of messages sent back and forth, and which are displayed at the bottom of your FTP app, in the form of a text message preceded by a number. "The number is intended for use by automata to determine what state to enter next; [only] the text is intended for the human user" according to rfc959. Nonetheless, most of the wording in such messages is obscure and so we wont spend much time on them, but there are plenty of resources on the web regarding commands, errors, and other FTP codes.

Some example server-to-client messages include:

426 Broken pipe.
500 Syntax error, command unrecognized.
501 Syntax error in parameters or arguments.
502 Command not implemented, superfluous at this site.
503 Bad sequence of commands.
504 Command not implemented for that parameter.

*UNIX TYPE L8*

When you log onto a server, one of the first things your client app asks the server for is its operating system platform and version for ease of interaction between the remote server and you, the local client. So, when you log on, your FTP app sends a SYST command.

"The command 'TYPE L 8' is often required to transfer binary data between a LOCAL machine whose memory is organized into (e.g.) 36-bit words and a machine with an 8-bit byte organization." [rfc969]

The most likely reason that the sysadmin doesnt want the actual server type (ex. SunOS 4.X) being displayed to anonymous users is for security reasons, for the most part. For example, each platform and version has a different set of weaknesses and thus some people scan looking for specific server types, which is why the type sometimes is not displayed. So when you see Unix TYPE L8 as the response to your client app`s automated SYST query, that relates to the file structure on that server and its ability to interact with a Unix, and not to the actual operating system of that particular server.

*User Commands*

To have the server carry out a task by remote control, you must instruct it to do so by using commands known to the server, i.e. FTP commands. Since graphical Windows-based FTP apps dont require the user to manually enter commands to use the app, they wont be discussed at length here.

Some example client-to-server commands include:

CWD Change Directory
PASV Passive Mode
STOR Store
STOU Store Unique
RNFR Rename From
RNTO Rename To
DELE Delete
RMD Remove Directory
MKD Make Directory
NOOP Anti-idle

*Types of Servers*

When you first log onto a remote server, it sends a message identifying the platform. There are basically two types of servers: Windows and Unix-flavor. The Windows NTs are the ones you want to use as a pub, and the Unix varieties (Unix, Sun, BSD) are used for gateways, to fill NTs (some Unix boxes are used for fast pubs if they cant support an xfer to an NT). One can use a remote server set up to operate as a 'Wingate' to then do an xfer from one NT to another NT, but we`ll leave Wingates aside.

*Permissions*

You`ll notice up to nine letters to the right of any file or directory. These are indicators of the perms that have been set by the sysop on that remote server. There are three types of users (admin, authorised user, and guest) and each has three degrees of permissions, on any given server, for a total of between 0 and 9 perms, termed as Read, Write, and Execute. You want to have perms for Read (downloading) and Write (uploading). If its not visually obvious, you`ll be able to determine later when you do a test file.

*Files and Folders*

The ability to write to the server and to navigate the folders (directories) depends on the perms set but if you encounter a directory that you cannot enter, that is because the sysop set it to 'protected' mode to stop guest users from accessing that specific dir. Other than sysop`ed protected dirs, there are certain other types of protected dirs, described in the text below.

*Exploring The Server*

Use SmartFTP and browse the server in search of prior 'occupants' because if there is recent pubbing activity already evident on that box, that means its taken so move on. SmartFTP will spot nearly any type of directory or file so be sure to drill down every path until you`re sure there is neither a tag nor a dir that clearly shows recent use (if 3 or 4 weeks old, the server really should be put to proper use). At that point you should tag it and then create a dir to hold your files.

*Making Your Pub*

The first thing you want to do is test the server for upload and download speed so upload a test 1MB file to the server, and then download it again and check the speeds. Once you are satisfied that its a useable pub, make a dir with your nick or group in its name and start your uploads. At some point the server might become filled so if that happens you should delete 10% of the total of the space occupied by your files. There has to be some free space left over for the remote server`s own operations.

*Creating Your Pub`s Contents List*
(for this, you`ll need a copy of UltraEdit)

Once the files are on the server, in FlashFXP click Directory/Copy Directory To Clipboard. Launch UltraEdit which opens a new document, and Paste the contents of your pub into the window in UltraEdit.

In UltraEdit, select Column/Column Mode. Now, click and drag your mouse sideways and up and down over the contents in the document so as to highlight the text that you want removed, and then right-click the text once its highlighted and choose Cut. Highlight whats left, right-click and choose Copy, and then paste that into a new text file on your desktop. Use that text file to store the addy and contents of your pub, and later you can Copy/Paste the now-edited contents of the pub into a post on a board.

You also can use one of the many pub-listing apps that now are available, including 'FTP List', among many others, which list the contents and path of your pub with a simple mouse-click.

To determine the size, and number of files and folders, in FlashFXP click Tools/CalculateSpaceUsed, and check both 'List Directories Recursively' and 'Do not Follow Folder Links'. FlashFXP will tally up the totals and you then can add that info in your pub post, i.e.:

Files: 10,212
Dirs: 2276
Space: 32.37GB

*Posting Your Pub*

When it comes time to post your pub on a board, be sure to use the HTML Code and include the [ and ] symbols. Use the example below but dont put a space after the first [ symbol. It was done here to avoid the example getting rendered as an HTML command.

[ code]IPandPathHere[ /code]

----- ----- -----

A Brief Introduction to FXP

Around the time of Windows NT Server`s release in the early 1990s, there appeared something called File Services Protocol or FSP (also called the 'F*cking Software Pirates'). FSP was a protocol that allowed a server to be set up to dish out copies of software and was for a time fairly popular. There also were FSP networks, and what were termed 'Trickle' servers or 'mirrors' that each held the same files and therefore allowed for more people to download files without stressing one central server. In theory, anyone could use an Archie archive search engine to locate files on an FSP Trickle server. FSP appears to be a forerunner to FXP or File eXchange Protocol, also known as File 'cross-transfer' Protocol.

The File cross-transfer Protocol (FXP) was based on the same principle, to carry out a transfer between two remote servers, conducted by a user at a third location. All that is required is that one of the boxes must be of the Unix-variety.

A Unix server can exchange files directly with an NT server because NT was specifically designed to interact with other varieties of servers at the time that Windows NT was developed (such as Unix, Sun/Solaris/SVR, and BSD). On the other hand, NTs were not designed to interact with other NTs in a similar manner and as a result one cannot generally send files straight from one NT to another NT under most conditions.

A person can use a Wingated NT server, a server that is running Wingate, which can route traffic in the same manner as a genuine Unix. But using a 'Wingate' isnt an especially fast method of doing an xfer for the most part, so we`ll focus here on genuine Unix-variety gateways.

Incidentally, as to 'non-FXPable' and 'FXP-able' servers, most servers can otherwise be set to facilitate a cross-transfer but are not, and the reason has to do with security. As it happens, one can carry out what is called a 'bounce attack' or other similar intrusive or damaging activities so if the server you`re using doesnt support an xfer, it likely is due to the sysop having set that server to not allow a bounce attack or any similar security threat, and since FXP requires the same route thru the server, cross-transfer also becomes disabled and so that box is termed 'non-FXPable'.

In order to copy files from a source to a target gateway, the user initiates a connection between the two servers which causes the one server to 'listen' on a port for the incoming files that are being transfered from the other server. It is otherwise identical to a regular FTP transfer to the casual observer, other than the fact that the source files reside on a remote server, and that they can be transferred from the one server to the other at speeds up to 2000k/sec or greater.

To get the files on the Unix in the first place, you need to have access to boards where there are plenty of people posting pubs. When you see a nice pub posted, you then simply copy the files off of that pub and onto your Unix-type server and then you can later send the files from your gateway to one or two NTs of your own. Finally, when you`re done copying the files to your own NTs, you then can post your pubs on some other boards, thereby spreading the files around to a greater number of people.

*Setting Your FXP Application*

You should obtain a working proxy for use when doing an xfer. The proxy is a remote server through which you are carrying out your actions, and it provides a degree of anonymity to you (of course anyone still could trace your IP if they really wanted to).

In FlashFXP, click Options/Preferences/Proxy-Firewall-Ident, and click Type and select SOCKS4. In the Host field, enter a working proxy that you`ve found on a web proxy site and tested or however you may find a working proxy, set the Port to 1080, and check the Passive Mode box. In Options/Preferences/Advanced make sure that you check 'Dont Leave Empty Directories'.

Now log onto a Unix and an NT and drag the files from the left pane to the right pane and the files will flow from the one to the other. The estimated speed is displayed after each file is copied over. Sometimes you`ll leave FlashFXP unattended while it does an xfer so to avoid having a dialog box open on your screen and hang your transfer while waiting for your response while you sleep or are away from your keyboard, set it ahead of time to deal with files that already exist on the target server. Select from the menu Options/When File Exists, and set it to Auto-Overwrite if smaller, and Auto-Skip for if same and if larger.

--- --- --- --- --- --- --- ---

*Trick Directories*

According to Microsoft, "Names cannot end with Dot '.', or Space ' '." Given that MS claims it cannot be done, it clearly is an unidentified exploit, or flaw, in Windows.

Windows uses a placeholder design for its path structure and so you can make a directory with an attached placeholder and then assign to that nested placeholder dir a name after the fact. This includes having a space in your parent locked dir`s name. If there is a space used after the name of the parent, the directory cannot be entered by clicking one`s way in.

*How to Make a Protected Directory*

Select Make New Directory in your FTP app and type:

/example[space]/[space]/

Then, select Make New Directory again and type:

/example[space]/path/

In order to click and enter such a directory, simply make another dir (a clone) with the same name in the same folder but without a space after the name, i.e.

/example/

Now, when you click the Protected Directory, /example /, it no longer is protected and you can click your way in.

To remove ONE OF YOUR OWN, EMPTY locked dirs, including illegal-name dirs, simply reverse the action used to make one, i.e.:

rmd /this[space]/example/

rmd /this[space]/[space]/

There are other dirs that have certain special properties, including adding one or more dot ( . ) characters. Again, according to Microsoft, this cannot be done, which simply is not true. For example, select Make New Directory and type:

/ . pub . / /

You cant make a clone dir on the vast majority of NTs for the clone will not display the dot at the end:

/ . pub . /

appears as:

/ . pub /

The Windows server will drop the last dot at the end when you try to clone it. If you use two dots, it`ll drop the last one only etc.

*Illegal/Reserved Names*

Windows does not normally allow a person to make a dir with the name CON, AUX, COMn (example, COM1, COM2, COM3, etc.), LPTn (example, LPT1, LPT2, LPT3, etc.), PRN, and NUL. But using the placeholder method one can make such a directory, i.e.

Select Make New Directory and type:

/nul[space]/[space]/

Select Make New Directory again and type:

/nul[space]/path/

Such an illegal named directory cannot normally be cloned so in order to view the directory`s contents, you need to either rename it or peek.

In order to unlock an illegal-named directory, use the RenameFrom/To raw command.

If the illegal dir is in the path:

/temp/lpt1[space]/pub[space]/here/

you would enter this raw command:

rnfr /temp/lpt1[space]/[space]/ and hit enter.

then when it asks you for the new name, type:

rnto /temp/aaa/ and hit enter.

You may have to rename/clone all the way down the path, but you`ll get there.

To make a proper clone:

Launch FlashFXP and log onto the server.
Get the path to the target, let say its:
/path/here/targetdir

Save this to a text file on your desktop:
/path/here/

Now launch BitBeamer, and locate the target. Right-click it, choose Rename, run your mouse over the name in the Rename box that opens, to highlight the name including any spaces, rightclick and choose Copy. Append that using Paste, to the path in the text file you made, i.e.
/path/here/whatever.you.just.copypasted.here/

Remember to put the / (slash) after you paste to your text file and be careful to add the slash where the cursor is blinking, i.e. after any spaces in the name.

Switch to FlashFXP, click Make New Dir, and paste into FlashFXP whatever you have in your text file. Go back to BitBeamer, hit Refresh, and you`ll see an exact clone was created and you can now click your way into the original target. With practice you can figure out how to do the opposite, i.e. how to gain the name of a dir that you want to remove. Remember if it ends in a dot, you may have to add a dot to your clone to copy the target accurately.

NOTE:
The list of reserved names on Windows98 includes many DOS Device Names (DDNs) as well, including 'XMSXXXX0' and other such DDNs. And, if you arrange the dirs in a certain order and then post a picture somewhere on the web pointing to it, or send someone an email or Word document pointing to the path, as soon as the person clicks your picture or HTML link, the target server will either crash right away or it will hang in limbo and crash after the kernel has become sufficiently unstable.

*Peeking*

One peeking method, discovered by a couple of fellows named goodslw and sung1st, devised for use in a DOS or Windows environment, uses a 'relative' reference:

i.e. if the target is called /COM1[space]/

Select ChangeDirectory to enter the placeholder:

/COM1[space]/[space]/

Then enter this raw command:

list -aF ../*/

Note: you dont really need to use -aF or any other flags, it simply makes some dirs display more easily for those who dont use Directory/ViewRawDirectory in FlashFXP.

You also could use this 'absolute' peeking method that I discovered this past summer:

list /COM1~1/*/

*Truncation*

Ironically, just this weekend I finally found information relating to the use of the tilde for truncation:
"To convert a POSIX or Win32 filename to a DOS-friendly filename, follow these steps:

1)Remove all Unicode characters
2)Remove all '.' but the last one if it is not the first character
3)Uppercase all letters
4)Remove forbidden characters
5)Truncate everything before the potential '.' to 6 characters, and add the string "~1"
6)Truncate everything after the potential '.' to 3 characters
7)[When] the name already exists, increment the string "~1" [i.e. ~2,~3, etc]"
linux-ntfs.sourceforge.net

Using either of the peeking methods cited above, you should followup the raw command by selecting Directory/ViewRawDirectory and copy/pasting out of that window, the name of the hidden nested directories. Just remember, if you get a 'syntax' error when trying to enter the peek command, just SAVE to file the peeking command, and then PASTE it into your Windows client app when you need to use the command. By pasting in the command, rather than typing it in manually, you can get past the syntax error.

*STOU*

The STOU method was first posted by goodslw in the same thread where he related the peeking method. I already had documented STOU, and peeking, but had refrained from posting, but now that it has been posted by others, it isnt necessary for me to not include the method in this revised tutorial.

A person can make a new dir on an NT5 and then type STOU and that directory cannot be renamed until the next time the host server is rebooted. Upon typing STOU, the client opens a channel with the server whereby the server then expects the client to send a file that is to be 'Stored Unique', and which the server demarks by creating a temporary and undeletable file that contains information pertinent to the file you`ve tricked the server into expecting. But the client doesnt send a file, you merely want to trick the server into waiting for it, meanwhile establishing a directory that remains "in process". Therefore, the parent dir cannot be renamed until the server is rebooted and the sequence is broken. But you can cause the files in your pub to become non-downloadable if you make more than one STOU .tmp file in the same path/directory.

"STOU ...asks the server to create a file under a new pathname selected by the server. The STOU parameter is optional; if it is supplied, it is a suggested pathname, which the server will ignore if there is already a file with that pathname. If the server accepts STOU, it provides the pathname in a human-readable format in the text of its response." D. J. Bernstein

The purpose for STOU was to allow two files with the same name to co-exist without the one overwriting the other:
"[STOU] sends a file to a remote site with an index number so that it will not overwrite another file with the same name that already exists there." http://www.ipswitch.com

*Locked Files*

In order to make a tag that cant be removed with the delete command, make a new text file on your desktop, launch FlashFXP, browse to your desktop, and browse to the server where you want to place the file, then upload it. After it is on the server, Rename it as:

yourname.yourgroup.date[space]/[space]/ and hit enter

You can however overwrite such a non-deletable file by making a dir with the same name, causing the dir to overwrite the tag file. This doesnt work on all NT servers.

Likewise, to make any file non-deletable, click Rename and do the same thing:

Example:

thefile.zip becomes

thefile.zip[space]/[space]/

To remove a file that you locked (made non-deletable), make a new text file on your desktop, upload it to the target dir, rename it exactly as the name of the file you want to remove, and hit OK. The 0-Byte file you uploaded deletes the 15MB non-deletable file by overwriting it in the MFT allocation, i.e. you end up with a bunch of 0-byte files in place of the gig of files that you had started with, allowing you to recycle the pub. As stated already, you can make a new directory and then name it exactly the same as the file you want to overwrite, and then remove the directory after it overwrites the locked file. But overwriting isnt supported on all NTs.

*Unusual Characters*

For those who dont know, when you see a dir with a '?' after its name, that apparently is a sign that the server is an NT-Mac hybrid, and that the sysop nuked your folder so any dir with a '?' at the end has been locked by a sysop using a Mac console, causing NT users to be unable to access the directory (the question mark is not allowed in Windows but there is no limitation on its use in Apple`s Mac platform nor most Unix servers).

If you encounter a directory with a " ; " in its name, you should use custom commands or else another client app to manually rename the dir. Different apps, including BitBeamer, SmartFTP, and FlashFXP, have problems with certain chars, including [, {, %, and many others. So, use custom commands or else use a client that wont stumble on client-specific characters.

--- --- ---

*Notable Error Messages*

I/O Overlap Operation in Progress

I first encountered this error about a year and a half ago and since that time I`ve sought background information on its cause, and only recently figured it out. I guessed it was an Inode design in NT, and though I still havent found confirmation, I did eventually find some useful info on Inodes in Unix.

"[F]iles are represented by inodes, directories are simply files containing a list of entries and devices can be accessed by requesting I/O on special files.

I-nodes:
Each file is represented by a structure, called an inode. Each inode contains the description of the file: file type, access rights, owners, timestamps, size, pointers to data blocks. The addresses of data blocks allocated to a file are stored in its inode. When a user requests an I/O operation on the file, the kernel code converts the current offset to a block number, uses this number as an index in the block addresses table and reads or writes the physical block."

And so, if you get the error I/O Overlap it apparently means that the total number of chars in your path has exceeded the 255 character limitation (known as Max_Path). So, rename your parent dirs until you have fewer chars in total in your path`s dir names, else you cannot move files in or out of any path that triggers the error.

Pipe Theft Detected

When you send data from pointA to pointB, you are said to be 'piping' the data. It is possible to jump out of the pipe`s intended course, for nefarious purposes, and when it is detected you get the Pipe Theft Detected error. If you get that error, its best not to hang around and ponder why.

--- --- ---

*List Flags*

"The user can obtain more information than is presented by default, by entering a 'raw' LIST command. Some list flags include:

-a include files and directories that begin with a dot (.). These are normally hidden files.

-d list directories by name rather than the contents

-F label directories with a trailing slash (/), executables with a trailing asterisk (*), and links with a trailing at-sign (@)

-l gives a long listing which includes access permissions,
owner, group, size in bytes, and time of last modification

-R recursively list the contents of subdirectories"
UNIX for Beginning Users, P-80 International Information Systems, September 16, 1991.

*ASCII*

Before I forget mentioning ASCII, last winter I noticed that quite a few ASCII charts include characters that cause problems for Unix or WindowsNT, and I suggested people should read over the charts and they would learn how to use such characters for directory-locking purposes. Essentially, many such characters appear identical to the user, since there arent enough 'display glyphs' to show all of the characters, for which reason many are assigned the same glyph. The simplest explanation is provided by a well-known ASCII expert named Roman Czyborra:

"Don't let your computer manufacturer tell you that there are ASCII characters beyond (the first 95 characters)... ASCII defined no standard for this and many manufacturers invented their own proprietary codepages.

"For its internal text encoding, MS Windows works in terms of its own character code... in addition it assigns displayable characters to some of the code points that ISO-8859-1 explicitly leaves undefined.

"The code points 0-31 and 127 are assigned to control characters in US-ASCII, not to displayable glyphs, and the ISO-8859-1 code continues this tradition, as well as declaring the range 128-159 inclusive to be reserved for unspecified control functions...As you might have noticed, some platforms (e.g PC, Mac) nevertheless use some of the code points in these ranges for displayable characters." Roman Czyborra

*Ghost Pub*

If you have a pub on which the LIST function is disabled and the files wont list, you can rely on a log on your drive generated by your FTP app so as to allow you to make reference to said files. Keep in mind that some ghost pubs wont list any contents, but some will list files and not dirs, or dirs and not files, or both, or neither. It all depends on the sysop but if the server wont list its contents, you can generate a log in FlashFXP that will contain the paths as you upload the apps, and which you can use later to access the apps.

Spasibo!

spasibo za poleznyu informaciu !

2 anton:
soglasno subju, msg dolzhna bit:

$> cd /pub
$> more beer

[This message has been edited by G_kOS (edited 24 December 2001).]