Editable Binary Database (.INI STYLE!)

bakekitsune · **Posted:** Wed Mar 19, 2008 6:51 am

This is my code, (it can save 1000 strings in 5 milliseconds WOOT!). Anyway that's all and good but i was just wondering if there was a better way of doing it? or a way to optimize it faster? any changes? its not quite finished yet...

You can also select which pieces of data from the file to load into memory.
Save a piece of data without having to save the whole file all over again.

Code:

    ttime1 = GetTickCount()

    Call SaveEditBin("bakekitsune", "data", "IAMDATA", 1000)
    
    ttime2 = GetTickCount()
    
    Call SaveEditBin("bakekitsune", "time", CStr(ttime2 - ttime1))
    
    MsgBox LoadEditbin("bakekitsune", "data", 543)

Code:

Sub SaveEditBin(Name As String, Var As String, Val As String, Optional ByRef ArraySize As Long, Optional ByRef nFileNum As Byte)
Dim i As Long, dbpos As Long

If nFileNum = 0 Then
nFileNum = FreeFile

    If ArraySize = 0 Then
        Open App.Path & "\" & Name & ".txt" For Binary Access Write Lock Read Write As #nFileNum
            Put #nFileNum, AscW(Var) * MAX_CHARS, sC & Var & "=" & Val
        Close #nFileNum
    Else
        dbpos = AscW(Var) + 255
        Open App.Path & "\" & Name & ".txt" For Binary Access Write Lock Read Write As #nFileNum
            For i = 0 To ArraySize
                Put #nFileNum, (dbpos + i) * MAX_CHARS, sC & Var & i & "=" & Val
            Next i
        Close #nFileNum
    End If
End If
End Sub

Function LoadEditbin(Name As String, Var As String, Optional ByRef ArrayNum As Long) As String
    Dim nFileNum As Byte, dbcompile As String, dbArray() As String

    If ArrayNum = 0 Then
        nFileNum = FreeFile
        Open App.Path & "\" & Name & ".txt" For Binary Access Read Lock Read Write As #nFileNum
            dbcompile = Space$(MAX_CHARS)
            Get #nFileNum, AscW(Var) * MAX_CHARS, dbcompile
        Close #nFileNum
    Else
        nFileNum = FreeFile
        Open App.Path & "\" & Name & ".txt" For Binary Access Read Lock Read Write As #nFileNum
            dbcompile = Space$(MAX_CHARS)
            Get #nFileNum, (AscW(Var) + ArrayNum + 255) * MAX_CHARS, dbcompile
        Close #nFileNum
    End If

dbArray() = Split(dbcompile, "=", MAX_CHARS, vbBinaryCompare)
LoadEditbin = dbArray(1)
End Function

Result: Text file with (you can edit with Notepad and it will come up with the right result)

Code:

time=05
data0=IAMDATA               
data1=IAMDATA               
data2=IAMDATA               
data3=IAMDATA
etc...x1000

In this case data543=IAMDATA if i change data543=OH!YEAH in notepad, then if you ask for data543 then you get OH!YEAH .

>>> PS: Props to Sepiroth for giving me my first lesson in binary ( Yes i optimized it with NO dope! or NOdoze... i always get confused between the two

EDIT: oh yeh kids don't take drugs its NOT cool)

Richardf · **Joined:** Sun Jul 30, 2006 4:00 pm **Posts:** 1230

On the topic of drugs. caffeine is in almost everything. Perhaps we can market caffeinated fruits? On the topic of the binary database...
When vbGORE was just a baby, it used binary files for everything. Problem was, it was the only source of that info. we had editors for them, but if you added something to the characters, or took something off of them(out of their types) when you went to load the binary info... it would just be.. broken, completely. So a few of the folks that were around back then begged and pleaded spodi to use a database. settled on mysql as you know.It's been both a blessing and a curse. On one hand It's easy to change info, add it, mix it, play with it, SEE IT. On the other hand the database can be a huge pain to setup. Theres pages dedicated to explaining HOW to install it.. and still folks have trouble with it. It's crazy! When it comes down to it. Both plagued dead, and tanar have settled on a hybrid system using the database to edit everything, then using binary files for some of the info the client needs instead of sending all those packets.

Now.. Going the ini way.. I can see it used for game settings, things folks can edit no problem which is why we just use the var_get and var_put. But npcs and objects would be in plain english, you would need to encrypt the files(still do it anyway but still) to keep kids from playing with them. (remember kids, never trust the client). And that would slow down the read time anyway.

bakekitsune · **Posted:** Wed Mar 19, 2008 12:10 pm

Quote:

On the topic of drugs. caffeine is in almost everything. Perhaps we can market caffeinated fruits?

lmao, fixing the problem by using it to your advantage... i like... i like

Anyway...

lol exactly hopefully this fixes all those problems

You can use the binaries as if they were .ini but they are as fast as binary.

You don't need an editor for them either, and if you delete something it still works as the position of the string isn't stored, just that deleted string will return 0. Basically its exactly the same as .ini... but binary

im not quite sure where your getting at lol anyway thanks for the reply

Sephiroth187 · **Posted:** Wed Mar 19, 2008 6:47 pm

Well, when storing strings that are apart of a type structure, they store with it an integer that helps determine the length of the string. Where as using it on a base level you have to define how big your string is, while its great using this method for storing and retrieving - the problem is if the user edits the notepad and maybe even delete a space here or there then the structure will mess up.

HOWEVER -

Code:

If nFileNum = 0 Then
nFileNum = FreeFile
...
end If

should just be

Code:

If nFileNum = 0 then nFileNum = Freefile

or even remove the arguement. Since it seems pointless when you can't access files that are already open.

bakekitsune · **Posted:** Wed Mar 19, 2008 7:57 pm

You know what i thought exactly the same thing until i did it by accident. I put an open/close script outside and forgot to delete the inside the procedure and it ended up being way faster than if i deleted the outside procedure lol fukd if i no maybe i did something werd.

Well so far ive tried deleting and inserting new objects i still get the right result lol im still trying but i did it coz calculating via assembly is way faster, and you can't delete the length or position of a string because its not stored but calculated via the string variable itself (hint hint assembly).

I also made it at base level so that people who use .ini (getvar, putvar) could migrate really easily because implemented in exactly the same way.

Sephiroth187 · **Posted:** Wed Mar 19, 2008 7:59 pm

IMO, I think using input line is more effecient in the case of dealing with strings.

bakekitsune · **Posted:** Wed Mar 19, 2008 8:01 pm

its not as fast lol no where near! trust me on this haha depends what you mean by efficient

Sephiroth187 · **Posted:** Wed Mar 19, 2008 8:09 pm

Well. I'd avoid using your method in 'time critical events' - since the way your going on about it, your using it in a time critical event. If I was after speed I'd generally make an array, cache them and based on a timer - store them in a file as temporary memory if it was unused.

bakekitsune · **Posted:** Wed Mar 19, 2008 8:14 pm

lol nah i meant my indexing method was faster (i dont know about my procedure as a whole, yours might be faster)

lol its not time critical ALTHOUGH

this thing stores at a rate of 1,000,000 (datai=iamdata) strings per second lol

therefore if there were 100players each with 100variables each that would take....

50 milliseconds

Anyway for absolute time critical events i have a temp full binary (numbers only) database, which is much faster.

Sephiroth187 · **Posted:** Wed Mar 19, 2008 8:20 pm

Depends on the value amount and size. If you were copying empty strings - sure.

In ASM I'd assume you'd just mostly have to transfer the data over through pointers right? Still - wouldn't the memory allocation take time. Try a stress test using the speed template and see how your method peforms there.

bakekitsune · **Posted:** Wed Mar 19, 2008 8:24 pm

look at the first post and my crude timing method with gettickcount()

Call SaveEditBin("bakekitsune", "data", "IAMDATA", 1000) <<<< NOT EMPTY STRINGS LOL!!!

Result came up with a txt file with 1000 strings saying data=IAMDATA and time=05

In ASM you use mov its basically all pointers, except much much faster than "copymemory"

Aight il try whats the speed template?

EDIT:Example:ASM can excecute the lenB() command in less than a millionth of a second, its just 5 lines of mov pretty much.

oh and thanks for the quick response

Spodi · **Posted:** Wed Mar 19, 2008 8:34 pm

bakekitsune wrote:

In ASM you use mov its basically all pointers, except much much faster than "copymemory"

Maybe since they (pointers and memory copying) are completely different functions? That'd be like saying, "When I run, I go faster when wearing shoes than if I eat tacos." :wink:

Sephiroth187 · **Posted:** Wed Mar 19, 2008 8:35 pm

GetTickCount isn't accurate. Use the timeGetTime and timeBeginTime functions (Just look in the VBGore source for the declares)

The speed template is somewhere in the wiki - I am not too certain where. When I'm finished doing this assignment I'll post a link.

copymemory isn't really a fast function, you can re-write in C++ faster, eg.

Code:

void __stdcall MyCopyMemory(int* dest,int* source,int length)
{
    for(int i =0;(i<(length/4));++i)
       dest[i] = source[i];
}

I've tested this method in C, even if you step over the memory bounds the program will not crash. *edit - only if its within the range of the int*

I'm not sure how its written out in ASM I haven't spent enough time - but if you could write a procedure in ASM that moves 32bits of memory at a single time that could be fast, then just write a TLB to write the declares - its even faster for DLL calling in Vb.

bakekitsune · **Posted:** Wed Mar 19, 2008 8:40 pm

omg REALLY! thanks! i just reference the .tlb instead of .dll yeh?

For me i would have to half the long then half the integer, until its down to bytes and then get the value, and then convert to ANSI, it would be too slow, i dunno how else you would do it. I just index by getting the first and last letter of the variable example "data1" the byte for "d" so index to the byte value of "d" then index to the byte value of "1"

EDIT: but the Space is so large that it doesn't matter if you delete something. Theoretically it shouldn't work lol, its something to do with VB's binary loading method.

Thanks man, id rather no C++ than NASM lol its more useful and better employment factor.

EDIT: That C++ function you did was many lines of ASM except the guy who did the ASM for C++ is probably 999x10^100000 times better than me therefore it would be faster using the C method.

bakekitsune · **Posted:** Wed Mar 19, 2008 8:43 pm

Spodi wrote:

bakekitsune wrote:

In ASM you use mov its basically all pointers, except much much faster than "copymemory"

Maybe since they (pointers and memory copying) are completely different functions? That'd be like saying, "When I run, I go faster when wearing shoes than if I eat tacos." :wink:

LOL! thats a really good analogy of it. I meant to say its like just using pointers you have to know the addresses. But your spot on there.

vbGore Free Online RPG Engine

Editable Binary Database (.INI STYLE!)