[Lazarus] Usage of Serial in a testing program?

[Bo Berglund via Lazarus]

I am writing a testing application for checking out a TCP to Serial
bridge component. I am using Lazarus 1.8.0 and fpc 3.0.4 on Windows 7

For the serial port handling I found the "built-in" Serial unit
described here:
http://wiki.freepascal.org/Hardware_Access#FPC_built_in_Serial_unit

Now I have a few questions about its usage:

1) Serial port name
-------------------
When opening the port the example does this:

str(ComPortNr,tmpstr);
ComPortName:= 'COM'+tmpstr+':';
serialhandle := SerOpen(ComPortName);

So for say port 29 this results in ComPortName 'COM29:'
Is this always the case? I have a hunch that for port numbers 1..9 the
Windows port naming is different than for higher port numbers...

And if I want to port this to Linux, I guess thta the name is
completely different, but no hint is given.

2) Blocking send?
------------------
Is this port blocking for sending?

The example has this:

status := SerWrite(serialhandle, s[1], writecount );

where s is a string holding the text to send (I will actually need to
use a TBytes array instead) and writecount returns the bytes actually
sent.

If I want to send an array of say 100 kbytes at 9600 baud, will the
call return only when the data have been completely sent or earlier?
It will take about 100 seconds to send that array...

3) Receiving data
----------------
For data reception I see this:

s:='';
ComIn:='';
while (Length(Comin)<10) and (status>=0) and not keypressed do
begin
status:= SerRead(serialhandle, s[1], 10);
...

Since the call parameters are the receive buffer and the count to read
I do not understand how this can work. s has been emptied (length = 0)
just before the call so how can SerRead stuff any data into the buffer
starting at s[1], which should not even exist???

I would like to have a way to read incoming data asynchronously so I
also could handle the user feedback and the TCP connection which is
sending the data that ultimately will arrive on the serial line...

Any suggestions on how to do this?

[Bo Berglund via Lazarus]

On Sun, 09 Sep 2018 13:33:22 +0200, Bo Berglund via Lazarus

Post by Bo Berglund via Lazarus
3) Receiving data
----------------

I forgot to add a question about read timeouts...

4) Read timeout
---------------
When running this to read incoming data:

status:= SerRead(serialhandle, s[1], 10);

How can I get out of the call if there never arrives 10 bytes? Is
there a timeout somewhere so that I can set it to some small value and
put the SerRead into a thread that can check arriving data and if
found fire off an event or such?

[Bo Berglund via Lazarus]

On Sun, 09 Sep 2018 16:26:29 +0200, Bo Berglund via Lazarus

Post by Bo Berglund via Lazarus
On Sun, 09 Sep 2018 13:33:22 +0200, Bo Berglund via Lazarus

Post by Bo Berglund via Lazarus
3) Receiving data
----------------

I forgot to add a question about read timeouts...
4) Read timeout
---------------
status:= SerRead(serialhandle, s[1], 10);
How can I get out of the call if there never arrives 10 bytes? Is
there a timeout somewhere so that I can set it to some small value and
put the SerRead into a thread that can check arriving data and if
found fire off an event or such?

Ok, I went to the sources and found this:

interface
...
{ Reads a maximum of "Count" bytes of data into the specified buffer.
Result: Number of bytes read. }
function SerRead(Handle: TSerialHandle; var Buffer; Count: LongInt):
LongInt;

implementation
...
function SerRead(Handle: TSerialHandle; var Buffer; Count: LongInt):
LongInt;
var
BytesRead: DWORD;
Timeouts: TCommTimeouts;
begin
if GetCommTimeouts(Handle, Timeouts) then
begin
Timeouts.ReadIntervalTimeout := MAXDWORD;
Timeouts.ReadTotalTimeoutConstant := 0;
SetCommTimeouts(Handle, Timeouts)
end;
if not ReadFile(Handle, Buffer, Count, BytesRead, nil) then
result := 0
else
result := BytesRead
end { SerRead } ;

So SerRead seems to read *up to* Count bytes, but does it do this by
checking what is currently stored in the operating system inbound
buffer or does it try to wait until Count bytes are available?

If so how is the timeout that is needed in such a case changed?
Seems to be set at infinity in the SerRead function (MAXDWORD)....
Or is it ReadTotalTimeoutConstant that is used during read?

If you drill further down you will get into ReadFile, but that is a
call into Windows API...

[Mark Morgan Lloyd via Lazarus]

I am writing a testing application for checking out a TCP to Serialbridge component. I am using Lazarus 1.8.0 and fpc 3.0.4 on Windows 7
For the serial port handling I found the "built-in" Serial unitdescribed here:http://wiki.freepascal.org/Hardware_Access#FPC_built_in_Serial_unit
str(ComPortNr,tmpstr);ComPortName:= 'COM'+tmpstr+':';serialhandle := SerOpen(ComPortName);
So for say port 29 this results in ComPortName 'COM29:'Is this always the case? I have a hunch that for port numbers 1..9 theWindows port naming is different than for higher port numbers...
And if I want to port this to Linux, I guess thta the name iscompletely different, but no hint is given.

I did quite lot of work on the serial.pp unit more recently than that
wiki page, in fact I didn't even know it existed. Please refer to the
comments in that file in the first instance.

[Bo Berglund via Lazarus]

On Sun, 9 Sep 2018 20:44:34 +0000, Mark Morgan Lloyd via Lazarus

Post by Mark Morgan Lloyd via Lazarus
I did quite lot of work on the serial.pp unit more recently than that
wiki page, in fact I didn't even know it existed. Please refer to the
comments in that file in the first instance.

Thanks Mark,
I did that too and there is stuff I don't find there, like:

Read blocking?
----------------------
From interface section:

{ Reads a maximum of "Count" bytes of data into the specified buffer.
Result: Number of bytes read. }
function SerRead(Handle: TSerialHandle; var Buffer; Count: LongInt):
LongInt;

Does this function read whatever is in the buffer when called up to a
maximum of Count, or does it actually wait until Count bytes have
arrived? If it waits, for how long?

Concerning the com port name for SerOpen():

{ Open the serial device with the given device name, for example:
\COM1, \COM2 (strictly, \\.\COM1...) for normal serial ports.
ISDN devices, serial port redirectors/virtualisers etc. normally
implement names of this form, but refer to your OS documentation.
Returns "0" if device could not be found }
function SerOpen(const DeviceName: String): TSerialHandle;

In the Wiki example they use COMx: where x is a number, but this is
probably only working for single digit numbers 1..9.
In any case the naming there is not described in serial.pp

I need to huse a high numbered com port since all I have is USB2Serial
dongles, which typically get you a port number like 29...

[Sven Barth via Lazarus]

Post by Bo Berglund via Lazarus
\COM1, \COM2 (strictly, \\.\COM1...) for normal serial ports.
ISDN devices, serial port redirectors/virtualisers etc. normally
implement names of this form, but refer to your OS documentation.
Returns "0" if device could not be found }
function SerOpen(const DeviceName: String): TSerialHandle;
In the Wiki example they use COMx: where x is a number, but this is
probably only working for single digit numbers 1..9.
In any case the naming there is not described in serial.pp
I need to huse a high numbered com port since all I have is USB2Serial
dongles, which typically get you a port number like 29...

Did you test it? Cause the serial unit only passes on the name to the
platform's API. So \COM29 should work for a Com port with number 29.

Regards,
Sven

[Mark Morgan Lloyd via Lazarus]

This post might be inappropriate. Click to display it.

[Giuliano Colla via Lazarus]

--

[Bo Berglund via Lazarus]

On Mon, 10 Sep 2018 13:20:35 +0200, Giuliano Colla via Lazarus

Il 09/09/2018 13:33, Bo Berglund via Lazarus ha scritto:<br>

Post by Bo Berglund via Lazarus
I am writing a testing application for checking out a TCP to Serial
bridge component. I am using Lazarus 1.8.0 and fpc 3.0.4 on Windows 7
For the serial port handling I found the "built-in" Serial unit
http://wiki.freepascal.org/Hardware_Access#FPC_built_in_Serial_unit
1) Serial port name
-------------------
str(ComPortNr,tmpstr);
ComPortName:= 'COM'+tmpstr+':';
serialhandle := SerOpen(ComPortName);
So for say port 29 this results in ComPortName 'COM29:'
Is this always the case? I have a hunch that for port numbers 1..9 the
Windows port naming is different than for higher port numbers...
And if I want to port this to Linux, I guess that the name is
completely different, but no hint is given.

Here is the procedure I'm using to fill up a combo box with the
names of the existing serial devices on the platform. Please note
the different naming conventions in Linux and Windows. Also note
that to report a serial port as existing a test is made to open it
(using the fpc serial function), and avoid "phantom" ports.

But does not this listing fail if a port is already in use, i.e. the
SerOpen() call will return 0 indicating it is not found?

procedure TForm1.btnScanPortClick(Sender: TObject);
var
PortNr: Integer;
PortName: String;
PortHandle: TSerialHandle;
begin
cbSelPort.Items.Clear;
cbSelPort.Text:= '';
{$IFDEF MSWINDOWS}
for PortNr := 1 to 9 do begin
PortName := 'COM' + IntToStr(PortNr);
PortHandle := SerOpen('\\.\'+PortName); //<= Fail if already open?
cbSelPort.Items.Add(PortName);
SerClose(PortHandle);
if cbSelPort.Text = '' then begin
cbSelPort.Text:=PortName;
PortSel.Caption:= PortName;
end;
end;
end;
{$ELSE}
for PortNr := 0 to 9 do begin
PortName := '/dev/ttyS' + IntToStr(PortNr);
PortHandle := SerOpen(PortName);
if cbSelPort.Text = '' then begin
cbSelPort.Text:=PortName;
PortSel.Caption:= PortName;
end;
cbSelPort.Items.Add(PortName);
SerClose(PortHandle);
end;
end;
for PortNr := 0 to 9 do begin
PortName := '/dev/ttyUSB' + IntToStr(PortNr);
PortHandle := SerOpen(PortName);
if cbSelPort.Text = '' then begin
cbSelPort.Text:=PortName;
PortSel.Caption:= PortName;
end;
cbSelPort.Items.Add(PortName);
SerClose(PortHandle);
end;
end;
{$ENDIF}
end;

This seems only to deal with ports 1..9, what about the higher
numbered ports 10..255?
And would it not be a long execution time for testing all 255 possible
com ports on Windows?

Post by Bo Berglund via Lazarus
2) Blocking send?
------------------
Is this port blocking for sending?
status := SerWrite(serialhandle, s[1], writecount );
where s is a string holding the text to send (I will actually need to
use a TBytes array instead) and writecount returns the bytes actually
sent.
If I want to send an array of say 100 kbytes at 9600 baud, will the
call return only when the data have been completely sent or earlier?
It will take about 100 seconds to send that array...

Yes, SerWrite is blocking.

Does this mean that it is actually waiting for the data to be
physically sent out on the wire or until the operating system transmit
buffer has absorbed all of the data?
If the latter then a possibility to resize the transmit buffer would
help. But how?

I hope you don't plan to send 100 kbytes
in a single write on a serial line. The probability of an error in
between is near to 100%!<br>
What I'm doing in such cases is to create a dedicated thread to
handle send and receive, without blocking the main thread, which
remains responsive, and split the send in many small blocks (and
getting some sort of ACK/NAK from remote receiver), which makes it
possible to show a progress bar or whatever visual feedback you
like.

Unfortunately not possible in the real case because the file transfer
protocol of the device only sends a header with the count of the data
to follow and then streams all of the bytes....

Post by Bo Berglund via Lazarus
3) Receiving data
----------------
s:='';
ComIn:='';
begin
status:= SerRead(serialhandle, s[1], 10);
...
Since the call parameters are the receive buffer and the count to read
I do not understand how this can work. s has been emptied (length = 0)
just before the call so how can SerRead stuff any data into the buffer
starting at s[1], which should not even exist???
I would like to have a way to read incoming data asynchronously so I
also could handle the user feedback and the TCP connection which is
sending the data that ultimately will arrive on the serial line...
Any suggestions on how to do this?

I never use SerRead, but rather SerReadTimeout (from the same
unit) which gives a much better control of what is going on. Of
course in that case your receiving buffer must be large enough to
accommodate the largest block you expect. (This replies also to
your further questions).

Did not see this before, but it seems like a good choice for reading
data in a loop until the return is less than Count, which means that
the timeout has caused the exit.
Thanks for the suggestion!

I hope that it helps.

Yes it does, but it would have been even better if not full of HTML
tags and stuff. I have edited out most but I guess there are some
still left...
For example inside the code there may be some things like &gt; rather
than >

[Giuliano Colla via Lazarus]

Post by Bo Berglund via Lazarus
This seems only to deal with ports 1..9, what about the higher
numbered ports 10..255?
And would it not be a long execution time for testing all 255 possible
com ports on Windows?

In my experience both Linux and Windows if you connect a new serial
device do set a port number just higher than the previous one already there.
This means that you'll have to deal with higher numbers only if you have
more than 9 serial lines in your platform, which is a bit unlikely. Not
to consider the likelihood of a platform with 200 serial lines!

[Bo Berglund via Lazarus]

On Mon, 10 Sep 2018 19:44:22 +0200, Giuliano Colla via Lazarus

Post by Giuliano Colla via Lazarus

Post by Bo Berglund via Lazarus
This seems only to deal with ports 1..9, what about the higher
numbered ports 10..255?
And would it not be a long execution time for testing all 255 possible
com ports on Windows?

In my experience both Linux and Windows if you connect a new serial
device do set a port number just higher than the previous one already there.

On Windows7 at least every time you attach a serial2usb dongle into a
different USB port than before it creates a new COM port with a higher
number. And this port is sticky on the PC too, it remains "taken".
So after a few years (my laptop is from 2011) it stacks up and even
though I have currently only attached 6 such converters (one is a
4-way) the com port numbers are 29-30-31-32-33-34...

Post by Giuliano Colla via Lazarus
This means that you'll have to deal with higher numbers only if you have
more than 9 serial lines in your platform, which is a bit unlikely. Not
to consider the likelihood of a platform with 200 serial lines!

I would eventually get there if I continue to unplug and re-plug these
dongles...

[Giuliano Colla via Lazarus]

Post by Bo Berglund via Lazarus
Unfortunately not possible in the real case because the file transfer
protocol of the device only sends a header with the count of the data
to follow and then streams all of the bytes....

I don't know how your application is built, but if the origin is an ftp
transfer, then data are sent in small packets. It would appear that you
somehow get the ftp data and send them to a serial line. It is at that
point of the processs that you should create your records to send them
individually via the serial line.

[Bo Berglund via Lazarus]

On Mon, 10 Sep 2018 19:49:39 +0200, Giuliano Colla via Lazarus

Post by Giuliano Colla via Lazarus

Post by Bo Berglund via Lazarus
Unfortunately not possible in the real case because the file transfer
protocol of the device only sends a header with the count of the data
to follow and then streams all of the bytes....

I don't know how your application is built, but if the origin is an ftp
transfer, then data are sent in small packets. It would appear that you
somehow get the ftp data and send them to a serial line. It is at that
point of the processs that you should create your records to send them
individually via the serial line.

No FTP!
What I meant is that the device I want to communicate with has a
proprietary serial protocol and it can do file transfers too.
The protocol is delimited and checksummed, but when a file is
transferred there is only one block of file data in the middle of the
transfer and it could be up to 1 Mbytes (the memory limit of the
device).

My application is meant to test WiFi<=>Serial converters to evaluate
those we could use with the data device. So I want to run data through
the unit under test in both direction to verify that it is going to
work in the real environment. In real use the data device is connected
by RS232 wire to the Serial to WiFi converter and a device controller
is hooked in via a TCP socket server on the converter.

The test application implements both endpoints since it aims to check
out the converter itself.

So I am dealing with one enpoint as a TCP socket (using Indy here) and
the other is the serial line where I am trying to use the Serial.pp
unit since it is built-in.
Once it is set up I will push a lot of data through and look at the
performance.

But now I will be off travelling for a week...

[Giuliano Colla via Lazarus]

Post by Bo Berglund via Lazarus

I hope that it helps.

Yes it does, but it would have been even better if not full of HTML
tags and stuff. I have edited out most but I guess there are some
still left...
than >

Sorry, I forced HTML for better readability.
But you may easily extract the clean text (without &gt and such) by
creating an empty html file (whatever.html), and opening it with
LibreOffice or OpenOffice. Then you may copy and paste into it the text
from the e-mail, and finally save it as a pure text fle (whatever.txt).
It is much faster and safer than manual editing.

Giuliano

[Bo Berglund via Lazarus]

On Sun, 09 Sep 2018 13:33:22 +0200, Bo Berglund via Lazarus
<***@lists.lazarus-ide.org> wrote:

Coming back to this issue after I have been travelling...

Post by Bo Berglund via Lazarus
For the serial port handling I found the "built-in" Serial unit
http://wiki.freepascal.org/Hardware_Access#FPC_built_in_Serial_unit
1) Serial port name
-------------------
str(ComPortNr,tmpstr);
ComPortName:= 'COM'+tmpstr+':';
serialhandle := SerOpen(ComPortName);
So for say port 29 this results in ComPortName 'COM29:'
Is this always the case? I have a hunch that for port numbers 1..9 the
Windows port naming is different than for higher port numbers...

I have now come as far in my application as I can test the way the
Serial unit opens and closes com ports.
It turns out that on Windows there are two port name syntaxes:

Ports 1..9: COM1 to COM9 will work
Ports 10..255: Only \\.\COM10 to \\.\COM255 will work

But one does not need to use different naming depending on the port
number, the second syntax \\.\COMx works also for ports 1..9

Notice that contrary to the wiki example adding a colon to the end
like COM32: does NOT work so that is a wiki error.

I have yet to complete other parts of the application so I don't yet
know if the data flow will wok as expected.

[Paul Breneman via Lazarus]

On 09/23/2018 02:39 AM, Bo Berglund via Lazarus wrote:
...

Post by Bo Berglund via Lazarus
Notice that contrary to the wiki example adding a colon to the end
like COM32: does NOT work so that is a wiki error.

If I remember correctly the colon at the end is required for WinCE?
--

[Bo Berglund via Lazarus]

On Sun, 23 Sep 2018 08:39:52 +0200, Bo Berglund via Lazarus

Post by Bo Berglund via Lazarus
I have now come as far in my application as I can test the way the
Serial unit opens and closes com ports.
Ports 1..9: COM1 to COM9 will work
Ports 10..255: Only \\.\COM10 to \\.\COM255 will work
But one does not need to use different naming depending on the port
number, the second syntax \\.\COMx works also for ports 1..9

Forgot to add a question regarding making this a cross-platform
program:

If I add a check for the platform inside my program, how sghould it
look like to work on both Windows and Linux?

Something like this:

function TWiFiCommTester.ConnectSerial(Port: byte; Baud: integer):
boolean;
var
ComportName: string;
Flags: TSerialFlags;
begin
FLastError := '';
{$IFDEF WINDOWS}
ComPortName:= '\\.\COM' + IntToStr(Port);
{$ENDIF}
{$IFDEF UNIX}
//What goes here?
ComPortName := ????
{$ENDIF}
FSerial := SerOpen(ComPortName);
....

[Mark Morgan Lloyd via Lazarus]

I have now come as far in my application as I can test the way the>Serial unit opens and closes com ports.>It turns out that on Windows there are two port name syntaxes:>>Ports 1..9: COM1 to COM9 will work>Ports 10..255: Only \\.\COM10 to \\.\COM255 will work>>But one does not need to use different naming depending on the port>number, the second syntax \\.\COMx works also for ports 1..9

If I add a check for the platform inside my program, how sghould itlook like to work on both Windows and Linux?
function TWiFiCommTester.ConnectSerial(Port: byte; Baud: integer):boolean;var ComportName: string; Flags: TSerialFlags;begin FLastError := ''; {$IFDEF WINDOWS} ComPortName:= '\\.\COM' + IntToStr(Port); {$ENDIF} {$IFDEF UNIX} //What goes here? ComPortName := ???? {$ENDIF} FSerial := SerOpen(ComPortName); ....

Sorry Bo and Paul, I mailed you directly when I meant to reply here.
This rolls up the messages from earlier.

Ports 1..9: COM1 to COM9 will workPorts 10..255: Only \\.\COM10

to \\.\COM255 will work

Which is something that I pointed out to you, although in my case I
think I only tested up to 12 (i.e. with an 8-port card).

What comes after 255: 0 or 1? If 0 which form or name does it require?

I have yet to complete other parts of the application so I don't

yetknow if the data flow will wok as expected.

Note that that unit was specifically written to be usable either with or
without separate communications threads. If you're /not/ using threads
then there's a callback so that the read-with-timeout functions can
periodically call Application.ProcessMessages, and that the places that
it is used are OS-specific. You obviously /don't/ want to use that
callback if the comms functions are in their own threads, since APM
should only be called from the main thread.

I'm probably losing this email address on Tuesday, if there's anything
that you think I can possibly clarify please raise it sooner rather than
later.

On 09/23/2018 02:39 AM, Bo Berglund via Lazarus wrote:...> Notice

that contrary to the wiki example adding a colon to the end> like COM32:
does NOT work so that is a wiki error.

If I remember correctly the colon at the end is required for WinCE?--

OK, but this is something that the application programmer needs to be
aware of. I suppose it would be possible to modify the unit so that if
it sees a trailing : it changes it to a leading \\.\ if there isn't one
there already, but since the exact names are OS-specific and since we're
already insisting that the application programmer or user puts in /dev/
for unix I'm not sure that's justifiable.

In any event I'm not in a position to do any mods right now and am
probably about to lose access to Solaris (which is probably still worth
testing against) permanently.

(* If there are no serial ports on the system then return NIL, otherwise
a *)
(* TStringList.
*)
(*
*)
(* This returns an object, it is the caller's (eventual) responsibility
to free *)
(* this.
*)
//
FUNCTION EnumeratePorts: TStringList;

(* On a Linux system with udev or similar the /dev directory will only
contain *)
(* devices which have been detected as being present. On an older system
it *)
(* will probably contain all possible devices so we need to restrict the
list *)
(* to what's reasonable; the user should still be able to enter a device
name *)
(* manually if necessary, e.g. where a machine with 2x standard ports
plus a *)
(* 2-port card insists that it's got ttyS0, ttyS1, ttyS45 and ttyS46.
*)

CONST countTtyS= 12; (* Main board plus an 8-port
card *)
countTtyUSB= 8; (* Four dual-port adapters
*)
countTtyI= 4; (* A single 4-port ISDN card.
*)

VAR searchRec: TSearchRec;
counter: INTEGER;

BEGIN
RESULT:= TStringList.Create;
RESULT.Sorted:= TRUE;
counter:= countTtyS;
IF FindFirst('/dev/ttyS*', faSysFile, searchRec) = 0 THEN
REPEAT
RESULT.Append('/dev/' + searchRec.Name);
DEC(counter)
UNTIL (FindNext(searchRec) <> 0) OR (counter <= 0);
FindClose(searchRec);
counter:= countTtyUSB;
IF FindFirst('/dev/ttyUSB*', faSysFile, searchRec) = 0 THEN
REPEAT
RESULT.Append('/dev/' + searchRec.Name);
DEC(counter)
UNTIL (FindNext(searchRec) <> 0) OR (counter <= 0);
FindClose(searchRec);
counter:= countTtyI;
IF FindFirst('/dev/ttyI*', faSysFile, searchRec) = 0 THEN
REPEAT
RESULT.Append('/dev/' + searchRec.Name);
DEC(counter)
UNTIL (FindNext(searchRec) <> 0) OR (counter <= 0);
FindClose(searchRec);
IF Result.Count = 0 THEN
FreeAndNil(RESULT)
END { EnumeratePorts } ;