AnyBoard Internals

The message data format

AnyBoard uses text files to store its data. The advantage of text files is that they are portable cross platforms. To move a forum from one server to another (might be different hardware, differemt operating system), we simply copy the files, same applies to backup process. If we are storing data in DBM files or a relational database, then all kinds of complexity arise. The protability of text files made them a popular choice of data format in the internet age.

AnyBoard uses the standard MIME format. MIME stands for Multipurpose Internet Mail Extensions and is defined by RFC2045. MIME is much faster to parse than XML, and it can even include binary data without encoding. Converting from MIME to XML is straightforward, in fact, AnyBoard has a function to do such conversion.

In AnyBoard, each message is identified by an integer sequence number, say 100, 101. The data of the messages are stored in the postdata/ directory with extension .dat, so for message 100, its data file is "postdata/100.dat". The data files contain the message fields as parts of a MIME file. There is also an index file, which contains the short fields of a message.

So the fields of a message are divided into two sets, one set are in both the index and the data files, and other set is in the data files only.

The fields of a message are:

  1. aK jE fI wW hC mM size pQ xE email eZz to mood tP rhost mtime scat track vers
  2. rlink_title img aliases body hack modifier
Field in 1) are listed also in the index file (.msglist), fields in 2) are in the message data file only.

Field name Meaning
aK the number of the top message of the thread, if the message itself is the top, this is its own sequence number
jE the number of the message's parent, 0 if this is a top message (no parent)
fI the seq number of the message itself
wW message subject
hC author name
mM posting time in UNIX time
size message size (bytes)
pQ IP address of the poster
xE a bitmap flag for indicating various states for this message
email email address of the author
eZz uploaded files, multiple file names are separated by single space (AnyBoard replaces spaces in orginal filename with _)
to recipient of the message, empty if it is not a private message
mood index of the emoticon
tP URL of the related link
rhost resolved domain name of the poster, empty if the feature is not enabled, or domain can be resolved
mtime last modification time in UNIX time
scat message category index
track cookie track
vers AnyBoard version
2) Fields in data file only
rlink_title related link title
img image URL
aliases aliases of the poster
body message body
hack not used
modifier modifier of the message
You can view the .dat files in the postdata/ directory to get an idea.

How to convert from other boards

  1. Create an new forum
  2. Convert the data files to AnyBoard format in the postdata/ directory
  3. Also create the .msglist index file
  4. Run the regenerate forum + individual message command from the admin panel of the forum

If you are using a paid version of AnyBoard, you don't need to do step 3), once you have the data files, you can use the recover command to generate the forum.

To contribute a converter, please email the program to support@netbula.com or post it at http//anyboard.net/netbula/support

Sample code to write AnyBoard data file

$ABDir = "/home/xxx/www/forum_dir/";

@mfs = qw(aK jE fI wW hC mM size pQ xE email eZz to 
          mood tP rhost mtime scat track vers);
@mfs2 = qw(rlink_title img aliases body hack modifier);

#The function below takes a hash which contains 
sub write_ab_msg{
    my %msg = @_;
    return if not $msg{fI}; #error, missing sequence number
    return if not $msg{aK};  #error, missing top sequence number
    open INDEX, ">>$ABDir/.msglist"; #append to index
    for(@mfs) {
	$msg{$_} =~ s/(\t|\n)/ /g; #make sure we don't have tabs in original data
    }
    print INDEX join("\t", @msg{@mfs}), "\n";
    close INDEX;

    open MSGDAT, ">$ABDir/postdata/".$msg{fI}.".dat" or die "Can't open file:$!";
    print MSGDAT "AB8DF\n";
    my $Bound="WWWCONV_SAGGSAGSAHAOSOSIQ".time().rand(); #just some random string
    $Bound =~ s/\.//g;
    print MSGDAT 
    "Content-type: multipart/form-data; boundary=$Bound\n",
    "Content-disposition: form-data; name=message_data$msg{fI}\n\n";
    for(@mfs, @mfs2) {
	print MSGDAT "--$Bound\n";
	print MSGDAT "Content-disposition: form-data; name=$_\n",
                   "Content-type: text/plain\n\n";
      print MSGDAT $msg{$_}, "\n";
    }
    print MSGDAT "--$Bound--\n";
    close MSGDAT;
}

#test
write_ab_msg(fI=>1, jE=>0, aK=>1, wW=>"hello world", hC=>"user1", body=>"Hello");
write_ab_msg(fI=>2, jE=>1, aK=>1, wW=>"jK:hello world", hC=>"user2", body=>"Hello",
	     email=>'ab@yahoo.com', mM=>time());

#####################END####################