MANUAL PAGE 1
Author: IanB - ian(at)i-asm.com
|
It does not need a registry entry for the yDLL.DLL file, but this MUST be present in the same installation folder as yDecode in order to provide all the critical high-speed CRC/MD5, repair and decoding routines.
yDecode may be installed anywhere, but for easiest interaction with NewsPlex it can be installed directly into the NewsPlex root directory, where the few files needed should not present a problem. yDecode will then correctly find the NewsPlex executable and begin using its \async folder as the default input path without further intervention, unless you are running NewsPlex with a separate work folder ("-d <dir>" startup option) in which case this will need to be set explicitly as the input path.
yDecode can be set to automatically startup and shutdown NewsPlex with itself, which avoids the possibility of Newsplex not terminating properly. It can also start NewsPlex at any time through a menu command if the correct path to it has been set. This requires any command line settings to be entered into the NewsPlex Settings dialog. (There will be few, if any, in 4.0+ installations as these are mostly incorporated in the etc\newsplex.ini file now). The "-d <dir>" option is NOT required in this setting, however, as the correct work path is deduced from the location of the \async folder and NewsPlex will be started up automatically in that correct folder.
yDecode uses a standard text INI file named yDecode.INI for loading its own startup options, rather than registry entries. The default decoding input and output folders, NewsPlex settings, filter settings and all other processing options may be set in the INI file as well as through the two Settings dialogs. It should be in the application folder so it can be parsed on startup, and it is rewritten fresh on every shutdown using the current option settings. Help on the filter rules is incorporated into the INI as comments to assist with manual alteration.
yCheck, packaged with previous versions of yDecode, is no longer required as the async processor for NewsPlex, to feed yDecode a stream of .YNC/.UUE files, so any previous references to it should be removed from the NewsPlex startup environment - ie. any "-X" option in the NewsPlex command line (NewsPlex 3.9 or earlier) or in the [async] settings in the etc\newsplex.ini file (for 4.0 plus).
To make the NNTP connection to NewsPlex, access via NNTP must be globally enabled, obviously. If running NewsPlex version 3.9 or earlier the "-s" command line option must NOT be used (this turns off NNTP access), otherwise for 4.0+ there should be a line in the "[access]" section of the etc\newsplex.ini file reading:
nntpEnabled=1yDecode must know any security settings that have been made for "admin-level" access to the NNTP and web connections (so that it can use the restricted NP* set of commands to control NewsPlex). If the \etc\newsplex.ini file is available (for 4.0+) then the user/pass info will be parsed automatically, but for 3.9 or earlier these were startup command line options which will need to be entered into yDecode's NewsPlex Settings dialog manually.
For security, these user/pass settings (if parsed successfully for 4.0+) are not shown in the NewsPlex settings dialog, and will NOT be used to authenticate the integrated user Telnet link. The user will have to authenticate themselves to NewsPlex manually through this link to access any restricted NNTP commands.
If they are in the correct application subfolder, the README and these manual files can be viewed at any time through the About tab of the Program Settings dialog.
In the main Program Settings dialog, splitfile and sourcefile deletion after successful processing can be set. There is also a simple option to automatically minimise yDecode while decoding.
Main input and default output folders are set here too. All NewsPlex integration is keyed to the input folder being tested as a valid NewsPlex \async folder, which is helpfully indicated here. Along with the filter rules settings, changes to these folders are implemented immediately while running from the subdialogs, and cannot be cancelled from the main Settings dialog once made there.
yDecode makes available space checks before starting any batch of decodes, and for safety will stop decoding if any of the specified output drives falls below the free space threshold set in the Program Settings dialog. It will only continue if files are removed on those drives. The threshold setting is any multiple of 10Mb, up to 10Gb.
Free space is also checked for split joining and repair operations, where space for both the source files and all the output files is required until finished. Additionally, a clear margin of 10Mb of free space on top will be needed. This is because these can be lengthy procedures and new data may be coming in all the time from NewsPlex or elsewhere.
yDecode pauses and offers a retry option in these circumstances which allows you to clear space as required for the procedure to continue, otherwise it will fail and will not be attempted again. It is therefore in your own interest that you monitor disk usage when downloading many or large binaries.
The final program options that can be set relate to verification. This can be switched off entirely, so yDecode acts as a "dumb" decoder and file router, or just the final repair functionality can be suppressed. This allows better response on extremely slow or memory-limited machines.
If both these options are enabled, removal of completed verification files can be done automatically on shutdown, when all files they reference have been fully verified, or the CRC/PAR/PAR2 files can be saved safely for reference.
In the NewsPlex Settings dialog, command line options for NewsPlex can be set, its home folder can be confirmed and NNTP link settings established if they cannot be parsed automatically from the \etc\newsplex.ini file.
If the NewsPlex home folder has been correctly set, yDecode can startup and shutdown NewsPlex safely at any time through its private NNTP back channel, if all the security settings are configured correctly. There are menu commands to do this, as well as settings to manage automatic startup and shutdown in synchronisation with yDecode itself.
If there is a NewsPlex link active, a separate user Telnet link to NewsPlex can be established at any time through another menu command, allowing full control over it. (See the NewsPlex manual for the full range of possible NNTP commands.) The size of the text window buffer is limited to 32K characters, so long replies from Newsplex cannot be viewed in full, but it does allow simple checks of the task queue and other configuration if correct authentication is provided.
The most useful NewsPlex setting is the option to automatically close all downloaded sets when the NewsPlex async queue is empty. From the user's point of view, this means that if all available source files have been selected in the newsreader, they have all been decoded and there is nothing left waiting to download, whatever is left either fully or partially decoded is the basis for repair to reconstruct the original fileset as posted.
Once the async queue is confirmed as empty, yDecode therefore calculates the total number of repair files needed for all open verification sets, and requests them itself through NewsPlex without user intervention unless there is no active NewsPlex link. This option is only useful if you know that all files for a set were queued by you in a single go. For downloads including large posts spread over many days the manual close action explained below would be a better choice.
There is a connected option, that allows "zombie" messages to be included when assessing the queue count. That would hold up processing until the unavailable messages reappear on some referenced server, but this might be a temporary glitch in the availability of a server or authentication. Ignoring "zombies" therefore allows faster verification and repair at the potential cost of the messages reappearing and being downloaded later. yDecode does not currently delete them from the NewsPlex task queue, so the user would need to do this manually.
In the User Actions dialog there are commands to close sets individually or globally, which allows manual initiation of this procedure at a time considered appropriate by the user. These also tell yDecode that the set(s) selected will not be receiving any further downloads and that it should therefore calculate and request the amount of repair files needed on the basis of whatever has been downloaded so far.
These two options will NOT be available, and neither will the automatic operation happen, if there are still files queued for decode or being processed or the Newsplex async queue tests as not empty. The manual commands can be initiated if there is no link, however, to allow incomplete downloads to be tested.
Also from the User Actions dialog, individual files can be imported directly into yDecode for verification, perhaps if they were downloaded as fills from a web Usenet archive rather than through Usenet directly. Verification files imported will also be parsed, and used for repair or to start new verification sets, so yDecode is never restricted to handling only files that it has decoded.
There are commands to view the current list of so-far unverified files and purge it if necessary. This can also be achieved by deleting the unverified.lst file in the \verification subfolder when yDecode is not running. Lastly, the current status of any multipart UUE files being processed can be viewed safely.
The datatype limitations in Visual Basic only allow a maximum integer size representing 2 Gigabytes, or 922 Terabytes if the Currency datatype is used. However, since all the VB file I/O statements also only support 2Gb file maximums, apparently, it seems rather pointless expecting this Visual Basic app to cope with single files or parts of multiparts larger than this, so I've not implemented any size workaround. This is due to VB6's FAT/Win95 heritage which was not updated for the capabilities of FAT32/NTFS in NT/XP.
However, this shouldn't be a huge problem as it is common Usenet practice to split or archive large multimedia etc. files into far smaller chunks of around 10Mb, which is well within the capabilities of this code. I've never seen a 2Gb file or part uploaded in a single piece yet, thankfully!
It will likely bomb for larger files that do get through, although it will safely refuse to concatenate splits where the size of the resulting file can be calculated and would be larger than 2Gb, thus presenting a problem. This would be most likely with full DVD images or other large video files.
The ongoing rewrite of the code in Assembler avoids this restriction by using direct API calls for all file I/O and assumes 64-bit filelengths throughout. These code portions were not used in this release, however. The next release will include smarter file-concatenation routines that will allow >2Gb joins.
In originally writing yDecode I assumed that no-one would want to create PAR2 files with ridiculously large block sizes (the recommended default in QuickPar is 384,000 bytes, ideal for the intended usage of Usenet posting) and therefore allocated memory freely on that basis. The PAR2 specs actually allow 64-bit block sizes, ie. 16Tb max! This version of yDecode will refuse to parse any PAR2 with a block size more than 500Mb, but even sizes approaching this will likely cause memory allocation failures and crashes if repairs are attempted. Similar problems would likely occur for PAR sets with large repair files.
Again, the forthcoming full Assembler rewrite will allow blocks of any legal size without using excessive memory allocations. For reference, the value of PAR2 is its ability to repair small portions of larger files, so the use of such excessively large blocks is not just impractical but counter-productive in the extreme.