Originally Posted by sepulchre

I mowed mine this morning. Got a little sun, and a little hay fever..

Hey sepulchre,

Couple questions (on programmatic optimization):

1.) If I don't need the FULL storage capabilities of the INTEGER data typecast, am I better off using something like "Short" (smallint) type, instead?

&

2.) If you look above, where I do the StringReplace command on the " symbols (quotes) above?

I do it 4x in a row manually!

This is because in Delphi (which I am pretty sure it is NOT the case in VB's REPLACE instruction), it only catches & replaces the first instance (& there is 4 quotes in place when I do an Access export of my HOSTS file data)

(Well, when I was using ACCESS (many other folks to makes HOSTS files do though I am certain of this in fact due to the presence of trailing spaces))

See, I quit on that because of excess trailing spaces on each line item/record in my HOSTS file that Access leaves...

SO, would I be better off doing a small 1-4 iterations FOR loop (fastest, but has to have number of times given it vs. other loop types like while etc.) as opposed to doing 4 literal stringreplace instructions?

* Thanks man... & hope your hay fever gets better!

APK

I installed Ubuntu and vista Business

Hi, Ian! Tell me how you like Vista business.

@apk:

1) It really depends on the database you're using and the situation. In most cases I would go ahead and use the Integer type unless you have space / speed constraints. That way your program doesn't need to decide which type to use. Also, 'Integer' is a good indicator that the data being stored is a number whereas Short could be a numeric value or a string, so Integer would make the source code more easily understood.
However, if the value will always fit into a Short and you do have limits on your space, or if large numbers of instances of the data are being read or written (a speed issue) then Short or SmallInt may the way to go.

2) First, is the replacement being done within the code by the program? Or is it in the source code? If your app will be doing the replacement then see 'For' versus 'While' . . . below.
If your doing the replacing in the source, well . . I have a copy of Delphi but have never used it. If its source code handler only finds and replaces the first instance searched for, well that just sucks. If possible, I would copy the source code and paste it into Notepad or Jarte or some similar little text handler and do the replace there. (I like Jarte - it's small, capable, and best of all, Free!) Then copy it back, unless Delphi uses weird characters that are hard to translate, which I doubt.


'For' versus 'While' loops Versus literal stringreplace instructions - This is hard to answer because I don't know the speed of your stringreplace instruction. You could do some benchmarks or perhaps there is documentation that has such details about the instruction set.
If you decide to use loops I would go with 'For' loops in most circumstances because the number of iterations only has to be determined once for each loop whereas a 'While' loop has to check to see if it's finished in each iteration. Depending on the situation that could slow execution considerably.

Also, just for future reference, in Powerbuilder there's a function that removes spaces, either preceding or trailing, called Trim. You might see if Delphi has something similar.

Hope you find this helpful!
Ken

Well, today I turn . . . . a year older. Let's leave it at that, ok?
Wish I could take all my old friends out for a drink but they're all 1000 miles away. Ah well, here's to them, and to all my friends here as well!

Originally Posted by sepulchre

Well, today I turn . . . . a year older. Let's leave it at that, ok?
Wish I could take all my old friends out for a drink but they're all 1000 miles away. Ah well, here's to them, and to all my friends here as well!

Happy birthday! Cheers to you

Thanks, Jack!

Originally Posted by sepulchre

@apk:

1) It really depends on the database you're using and the situation

Not a DB, just a text file (HOSTS files are only that)

Originally Posted by sepulchre

Also, just for future reference, in Powerbuilder there's a function that removes spaces, either preceding or trailing, called Trim. You might see if Delphi has something similar.

I pull an "interesting trick" for that one (Removing that trailing blank)

... & that is to do a C/C++ style StrRev (ReverseString in Delphi) & THEN, that makes the trailing blank the 1st character & guess what? That TAKES ADVANTAGE of StringReplace's "weakness" of only working on 1st occurance of what is sought to be replaced... in this case? The "trailing blank", & voila!

THEN - Reverse it again: IF you have a leading blank? StringReplace catches THAT one, right then too!



ON TRIM: Delph has it, I use it & also has LeftTrim/RightTrim stuff (plus analogs to Midstring in VB, etc.)

In fact, you can see that in the code above (on last page, I put up the sourcecode to my app for this purpose in discussion with you, & in case anyone here is interested in that kind of thing)!

(OR, just see my "p.s." below (editing it in now, so you don't have to scoot back to the page prior to this one))

Originally Posted by sepulchre

In most cases I would go ahead and use the Integer type unless you have space / speed constraints

On space ones:

I do, & decided I can't use SmallInt (SHORT) in my code...

I.E.-> I have over 533,000 entries in my HOSTS file (unique ones) & smallint/short only stores 32,767...

No point in using Short (smallint) on my COUNTER variable (I use this to tell users which line of the file it is working on - see sourcecode on last page prior to this one (FORGET THAT (EDITING NOW) - see my P.S. below).

HOWEVER - I am concerned about speed...

How fast I can get this program (sourcecode on last page) to tear thru that file (533,000++ entries in it now) is of import to me... right now, as is, I can do that many entries in just under 2 minutes time, pretty fast... but, always looking for more speed.

E.G.-> By way of comparison? Doing this in notepad.exe, via its EDIT menu, REPLACE command?? Would take DAYS!

I think I will have to stay integer though (for the counter variable) since smallint/Short datatype cast limits me to 32,767 entries only... can't have that, as I have far more than that, in HOSTS files line item entries.

Originally Posted by sepulchre

That way your program doesn't need to decide which type to use.

I never, EVER, use "Variant" datatype, unless I can't avoid it... & I always typecast vars, STRICTLY!

(Fact is, I have to in Delphi - & in VB, I do the "option explicit" for the reasons of gaining speed in avoiding variant resolution of data type being operated on).

Originally Posted by sepulchre

Also, 'Integer' is a good indicator that the data being stored is a number whereas Short could be a numeric value or a string

I'm pretty sure that in Delphi @ least, Short IS strictly a numerical datatype though, on this account. Still, I am steering clear of it, per the reasons above (noting the 32,767 limit on short/smallint, & I have FAR MORE entries @ 533,000++ in my HOSTS files... & my "counter" variable needs more than Short/SmallInt give me)

Originally Posted by sepulchre

so Integer would make the source code more easily understood. However, if the value will always fit into a Short and you do have limits on your space, or if large numbers of instances of the data are being read or written (a speed issue) then Short or SmallInt may the way to go.

Right, agreed 110%... but again, where I was considering its use? I can't use it... had to look @ it again & saw that much.

Still, I thought I would ask anyhow, for future scenarios where I do have totals less than 32,767 (smallint/Short addressability limit).

Originally Posted by sepulchre

2) First, is the replacement being done within the code by the program? Or is it in the source code?

On the data in the HOSTS file.

Originally Posted by sepulchre

If your app will be doing the replacement then see 'For' versus 'While' . . . below.

That's rehashing what I wrote above, so, I guess I was correct about that much... & IF I were to do this? I would use a FOR loop (since it is KNOWN to be FASTER, but... you have to have a finite known number of iterations you want to perform... & I do - Access applies 4 quotes around every entry consistently in my HOSTS files on exports, & thus, I do it 4x).

My main question was this:

Doing each replace manually, literally in the same StringReplace instruction issued 4x in a row... is this MORE, or LESS, efficient than doing a FOR LOOP with 1-4?

Also?

I'd be almost willing to bet the compiler optimizes that into a FOR LOOP... w/ out me having to code it as such.

Originally Posted by sepulchre

If your doing the replacing in the source, well . . I have a copy of Delphi but have never used it. If its source code handler only finds and replaces the first instance searched for, well that just sucks.

Well, it'll do the job... just not as nice as VB... VB has the best string handling library commandset I've ever used in fact, a plus in its favor (but, it is far slower too, far FAR slower - especially in string processing, which is what this program IS about... heck, even MSVC++ is 2x as slow as Delphi in both STRINGS and MATH work, just so you know)

Originally Posted by sepulchre

This is hard to answer because I don't know the speed of your stringreplace instruction.

Well, based on tests done in Visual Basic Programmer's Journal Sept./Oct. 1997 issue "INSIDE THE VB5 Compiler"? Delphi knocked the chocolate out of BOTH MSVB & MSVC++ in both STRINGS & MATH work... by orders of magnitude no less... more than 2-5x as fast in fact.

Also, considering that Yes, VB's REPLACE instruction does do a better job by replacing ALL instances of a string in a character array (string)? It's going to be slower, just on that note... plus, the fact that VB is slower period, in string processing than Delphi is.

Originally Posted by sepulchre

You could do some benchmarks or perhaps there is documentation that has such details about the instruction set.

Pretty much have, just by what I noted above.

Originally Posted by sepulchre

If you decide to use loops I would go with 'For' loops in most circumstances because the number of iterations only has to be determined once for each loop whereas a 'While' loop has to check to see if it's finished in each iteration. Depending on the situation that could slow execution considerably.

And, the WHILE loop has to also check/recheck the TOTAL number of elements after each loop iteration run, too afaik (& if it DOESN'T? I'd be MIGHTY surprised in fact, because THAT, would be a hugely "bad move")... slowing it up.

FOR demands it be given a SPECIFIC number of iterations (thus, you have to know the total number of elements to process, & I do, 4 quotes from Access exports of HOSTS files after normalization), but is FAR faster as I noted above.

Originally Posted by sepulchre

Hope you find this helpful!
Ken

I did, thanks, & I hope you find my response material useful too.

APK

P.S.=> Ken/Sepulchre - FOR YOUR REFERENCE:

Code for my "APK Hosts File Rewriter 1.0++" is here (small, but F A S T):

----

unit Unit1;

//HAND & COMPILER LEVEL OPTIMIZATIONS SWITCHES/#PRAGMA DIRECTIVES
{$define nodebug}
{$STACKCHECKS OFF}
{$D-}//will prevent placing Debug info to your code.
{$L-}//will prevent placing local symbols to your code.
{$O+}//will optimize your code, remove unnecessary variables etc.
{$Q-}//removes code for Integer overflow-checking.
{$R-}//removes code for range checking of strings, arrays etc.
{$S-}//removes code for stack-checking. USE ONLY AFTER TESTING !
{$Y-}//will prevent placing symbol information to your code
{$H-}//Forces ShortString usage (no heap usage)... apk

interface

uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls, Buttons, ComCtrls, StrUtils;

type
TForm1 = class(TForm)
OpenDialog1: TOpenDialog;
BitBtn1: TBitBtn;
StatusBar1: TStatusBar;
Label1: TLabel;
Label2: TLabel;
Label3: TLabel;
Label4: TLabel;
procedure BitBtn1Click(Sender: TObject);
procedure FormCreate(Sender: TObject);
private
{ Private declarations }
public
{ Public declarations }
end;

var //GLOBAL
Form1: TForm1;

implementation

{$R *.dfm}
{$H-}
{$h-} //FORCE SHORTSTRING USAGE (below 254 char) so it does not use stack, & keeps memory usage on local heap (faster)... apk

procedure TForm1.BitBtn1Click(Sender: TObject); register; //use register loading, equivalent of FastCall C/C++ stuff, & faster... apk
var
Save_Cursor:TCursor;
F1: TextFile;
F2: TextFile;
Ch: ShortString; {I can & did do this here, vs. String datatype cast (no url I use is over 110 characters + it matches the $h- compiler directive for optimization)... apk}
Flags: TReplaceFlags;
Counter: Integer; {Possible optimization: Use smaller numerical datatype, since I doubt I'll use the full capabilities of INTEGER in THIS program that works on a HOSTS file... apk}
begin inherited;
Save_Cursor := Screen.Cursor;
Screen.Cursor := crHourGlass;
Ch:= Trim('');
Counter:=0;
Label3.Caption:= Trim(DateTimeToStr(now));
Label3.Update;
Label2.Refresh;
Label4.Caption:= Trim('Waiting to complete...');
Label4.Update;
Label4.Refresh;
//Time slice - internally multitask here... apk
Application.ProcessMessages;
{ACTUAL "ACTIVE INGREDIENT" (code engine that does the actual work) HERE}
//==============================================
try
if OpenDialog1.Execute then
begin
If OpenDialog1.Filename <> '' then
begin
BitBtn1.Enabled:=False;
AssignFile(F1, OpenDialog1.Filename);
Reset(F1);
AssignFile(F2, 'B:\HOSTS\NewNormalizedFinalMinusTrailingBlanks.tx t');
Rewrite(F2); //Blanks out, no need for FileUtils unit If FileExists... apk
while not Eof(F1) do
begin
Counter:=Counter+1;
Readln(F1, Ch);
//Replace ANY 127.0.0.1(tab) with 0.0.0.0(space) 1st... apk
Ch:= StringReplace(Ch, '127.0.0.1', '0.0.0.0', Flags);
//Reverse the string first, & StringReplace will nail the trailing blank
Ch:= ReverseString(Ch);
Ch:= StringReplace(Ch, ' ', '', Flags);
//Reverse it again & write it (trailing blank is removed now)... apk
Ch:= ReverseString(Ch);
Ch:= StringReplace(Ch, '0.0.0.0', '0.0.0.0 ', Flags);
Ch:= StringReplace(Ch, '0.0.0.0 ', '0.0.0.0 ', Flags);
//Burn quotes from AccessExport (after normalize filedata) to text... apk
Ch:= StringReplace(Ch, '"', '', Flags);
Ch:= StringReplace(Ch, '"', '', Flags);
Ch:= StringReplace(Ch, '"', '', Flags);
Ch:= StringReplace(Ch, '"', '', Flags);
Writeln(F2, Ch);
Label4.Caption:= Trim(DateTimeToStr(now));
Label4.Update;
Label4.Refresh;
StatusBar1.SimpleText:= Trim(IntToStr(Counter) + ' ' + Ch );
StatusBar1.Update;
StatusBar1.Refresh;
end;
CloseFile(F2);
Flush(F2);
Application.ProcessMessages;
end;
CloseFile(F1);
Flush(F1);
Application.ProcessMessages;
end;
//==============================================
except //ERR TRAPPING VIA EXCEPTION/RAISE EXCEPTION/CUSTOM ERRHANDLER I BUILT HERE... apk
begin
Application.ProcessMessages;
//Time slice-multitask internally here, after loop is done, even after Exception catch on err/abend... apk
BitBtn1.Enabled:=True;
Screen.Cursor := Save_Cursor;
Screen.Cursor:=crHourGlass;
close;
end
end;
Application.ProcessMessages;
//Time slice-multitask internally here, after loop is done... apk
BitBtn1.Enabled:=True;
Screen.Cursor := Save_Cursor;
Screen.Cursor:=crDefault;
Application.ProcessMessages;
//Time slice-multitask internally here, after loop is done... apk
Application.MessageBox(PChar('Done: Total lines processed = ' + IntToStr(Counter)),nil);
end;

procedure TForm1.FormCreate(Sender: TObject); register; //use register loading, equivalent of FastCall C/C++ stuff, & faster... apk
var
ProcessID: DWORD;
ProcessHandle: THandle;
ThreadHandle: THandle;
begin inherited;
{RUNNING @ REALTIME CPU PRIORITY FOR UTMOST IN SPEED via Win32 API call usage for that below... apk}
ProcessID:= GetCurrentProcessID;
//Application.Handle SHOULD do this as well here... apk
ProcessHandle:= OpenProcess(PROCESS_SET_INFORMATION,
false, ProcessID);
SetPriorityClass(ProcessHandle, REALTIME_PRIORITY_CLASS);
ThreadHandle:= GetCurrentThread;
SetThreadPriority(ThreadHandle, THREAD_PRIORITY_TIME_CRITICAL);
end;

end.

{COPYRIGHT APK - Alexander Peter Kowalski 06/01/2008 onwards}

----

apk

Originally Posted by sepulchre

Well, today I turn . . . . a year older. Let's leave it at that, ok?

Happy Birthday, Ken!!!

Dave

Originally Posted by DaveMo~

Happy Birthday, Ken!!!

Dave

YES! Where are my manners?



HAPPY B-DAY SEPULCHRE!

APK

Thanks guys! I'm stuffing myself with my favorite: spice cake!