[Lazarus] Searchable docs online

[Michael Van Canneyt via Lazarus]

Hello,

Given the questions of late, I have created a search page for the docs:

https://www.freepascal.org/docsearch/docsearch.html

This will search in the manuals, and in the reference documentation for the
units.

There is ample room for improvement, but the page already works.

Things to improve:
- Allow 'contains'
- Allow 'minimum rank' (rank is the number of times a word appears in a page)
- Allow to specify in which manual you want to search.
- Integrate searching in the doc pages themselves.

The underlying mechanisms for this search are 100% FPC based:
fpindexer, fcl-web, and the webpage logic is done using pas2js.
(meaning you need to have Javascript enabled)

Feedback and comments welcome.

Michael.
--

[Marcos Douglas B. Santos via Lazarus]

On Thu, Jul 5, 2018 at 10:50 AM, Michael Van Canneyt via Lazarus
<***@lists.lazarus-ide.org> wrote:
>
>
> Feedback and comments welcome.

There is no `action` in the search form. Nothing happens...

Regards,
Marcos Douglas
--

[Michael Van Canneyt via Lazarus]

On Thu, 5 Jul 2018, Marcos Douglas B. Santos via Lazarus wrote:

> On Thu, Jul 5, 2018 at 10:50 AM, Michael Van Canneyt via Lazarus
> <***@lists.lazarus-ide.org> wrote:
>>
>>
>> Feedback and comments welcome.
>
> There is no `action` in the search form. Nothing happens...

Of course there is no action.
The Javascript catches the 'OnClick' and executes the request in the
background.

Michael.
--

[Marcos Douglas B. Santos via Lazarus]

On Thu, Jul 5, 2018 at 2:47 PM, Michael Van Canneyt via Lazarus
<***@lists.lazarus-ide.org> wrote:
>
>
> On Thu, 5 Jul 2018, Marcos Douglas B. Santos via Lazarus wrote:
>
>> On Thu, Jul 5, 2018 at 10:50 AM, Michael Van Canneyt via Lazarus
>> <***@lists.lazarus-ide.org> wrote:
>>>
>>>
>>>
>>> Feedback and comments welcome.
>>
>>
>> There is no `action` in the search form. Nothing happens...
>
>
> Of course there is no action. The Javascript catches the 'OnClick' and
> executes the request in the
> background.

I know that... but was not working. I've tried a few times and nothing happened.
Now, it works.

Regards,
Marcos Douglas
--

[Michael Van Canneyt via Lazarus]

On Thu, 5 Jul 2018, Marcos Douglas B. Santos via Lazarus wrote:

> On Thu, Jul 5, 2018 at 2:47 PM, Michael Van Canneyt via Lazarus
> <***@lists.lazarus-ide.org> wrote:
>>
>>
>> On Thu, 5 Jul 2018, Marcos Douglas B. Santos via Lazarus wrote:
>>
>>> On Thu, Jul 5, 2018 at 10:50 AM, Michael Van Canneyt via Lazarus
>>> <***@lists.lazarus-ide.org> wrote:
>>>>
>>>>
>>>>
>>>> Feedback and comments welcome.
>>>
>>>
>>> There is no `action` in the search form. Nothing happens...
>>
>>
>> Of course there is no action. The Javascript catches the 'OnClick' and
>> executes the request in the
>> background.
>
> I know that... but was not working. I've tried a few times and nothing happened.
> Now, it works.

I will add some feedback if there are no results.

Michael.
--

[Vojtěch Čihák via Lazarus]

Hi,
 
what is your browser? I use Falkon, here works both button and enter-key.
 
V.
______________________________________________________________
> Od: "Marcos Douglas B. Santos via Lazarus" <***@lists.lazarus-ide.org>
> Komu: Lazarus mailing list <***@lists.lazarus-ide.org>
> Datum: 05.07.2018 18:59
> Předmět: Re: [Lazarus] Searchable docs online
>
On Thu, Jul 5, 2018 at 10:50 AM, Michael Van Canneyt via Lazarus
<***@lists.lazarus-ide.org> wrote:
>
>
> Feedback and comments welcome.

There is no `action` in the search form. Nothing happens...

Regards,
Marcos Douglas
--
_______________________________________________
Lazarus mailing list
***@lists.lazarus-ide.org
https://lists.lazarus-ide.org/listinfo/lazarus <https://lists.lazarus-ide.org/listinfo/lazarus>

[Ryan Joseph via Lazarus]

> On Jul 5, 2018, at 7:50 AM, Michael Van Canneyt via Lazarus <***@lists.lazarus-ide.org> wrote:
>
> https://www.freepascal.org/docsearch/docsearch.html

Great start, I would have used it this morning if I knew it was here.

1) where is the source? I’ve never seen a FPC CGI program and I’m curious/willing to contribute.

2) The sorting isn’t what I’d expect. For example search for “TFPGObjectList” and I would expect to see the class reference as the top result (tfpgobjectlist.html) but instead it’s lower down and preceded by methods. Can this be fixed?

Regards,
Ryan Joseph

--

[Michael Van Canneyt via Lazarus]

On Thu, 5 Jul 2018, Ryan Joseph via Lazarus wrote:

>
>
>> On Jul 5, 2018, at 7:50 AM, Michael Van Canneyt via Lazarus <***@lists.lazarus-ide.org> wrote:
>>
>> https://www.freepascal.org/docsearch/docsearch.html
>
> Great start, I would have used it this morning if I knew it was here.
>
> 1) where is the source? I’ve never seen a FPC CGI program and I’m curious/willing to contribute.

I will commit it tomorrow and post the location.

>
> 2) The sorting isn’t what I’d expect. For example search for “TFPGObjectList” and I would expect to see the class reference as the top result (tfpgobjectlist.html) but instead it’s lower down and preceded by methods. Can this be fixed?

As said, the ranking is based on the number of occurrences of the word in a
page. The search engine has no semantic knowledge of the structure of the
documentation, it is a simple text search engine.

But conceivably, other mechanisms can be added to change the ranking.

Mihcael.

[Anthony Walter via Lazarus]

Michael,

Thanks for you work. You might want to refer to my doc site after I finish
tiling and painting my server room.

https://docs.getlazarus.org

It implements pretty much the same thing with a few differences:

- Search is incremental with a drop down list that populates as you type
(XMLHttpRequest).
- Search rank is adjusted by most frequent clicks, with more clicks
ranking higher.
- Properties, methods, and events link two ways to a side panel, rather
than a floating window.
- RTL, FCL, and LCL packages can be selected from a drop down list.

[Michael Van Canneyt via Lazarus]

On Thu, 5 Jul 2018, Anthony Walter via Lazarus wrote:

> Michael,
>
> Thanks for you work. You might want to refer to my doc site after I finish
> tiling and painting my server room.
>
> https://docs.getlazarus.org
>
> It implements pretty much the same thing with a few differences:

I assume this is all implemented in Pascal ?
It would be better if you made available the technology that underpins all that.

Michael..
--

[Anthony Walter via Lazarus]

Michael,

It's mostly Typescript, and the source is on the page (F12 developer tools,
source map is automatically linked). The Typescript detects key presses,
resets a short timeout, invokes XMLHttpRequest, and rebuilds the drop down
using a template. The server part that response to XMLHttpRequest is based
on a simple IHttpHandler HandleRequest method that just responds with the
same HTML you are using.

[Michael Van Canneyt via Lazarus]

On Fri, 6 Jul 2018, Anthony Walter wrote:

> Michael,
>
> It's mostly Typescript, and the source is on the page (F12 developer tools,
> source map is automatically linked). The Typescript detects key presses,
> resets a short timeout, invokes XMLHttpRequest, and rebuilds the drop down
> using a template. The server part that response to XMLHttpRequest is based
> on a simple IHttpHandler HandleRequest method that just responds with the
> same HTML you are using.

Typescript on a page about lazarus ?
pas2js is the way to go these days ;-)

I was not talking about the autocomplete, this is easily handled.
I will add that later on, after all it's just a first version what I put online...

I'm more interested in the underlying docs structure. I've been thinking
about adding an option to fpdoc to create a HTML version using frames, it
would save me some work if I could reuse the work you did.

Well, never mind.

Michael.

(BTW your server is not responding currently)
--

[Michael Van Canneyt via Lazarus]

On Sun, 8 Jul 2018, Ryan Joseph wrote:

> Just tried to use https://www.freepascal.org/docsearch/docsearch.html and it seems to have a bug (not working, errors in the JS console). Awaiting the source on svn also. :)

Possibly, I have been doing some experiments. I have re-enabled it.

It is now also referred to from the main docs page:
https://www.freepascal.org/docs.var

The search term edit box now has type-ahead enabled.

Sources (webpage/server) can be found on:
https://svn.freepascal.org/svn/html/docsearch/

The server needs the latest fpindexer:

https://svn.freepascal.org/svn/fpc/trunk/packages/fpindexer/

The indexer works using postgres, but sqlite, firebird (in fact any SQLDB DB) are
also supported. There is also support for a custom file format.

The "client" is written in pas2js.

So it comprises now a full-fledged example of how FPC can be used as a
full-stack web development environment.

In a couple of moments, I will commit a more general version of the
HTML searchengine under fpindexer examples.

Michael.
--