Disabling searching in content does not work

[joerg.hambuch]

Hello,

in topic http://www.oxygenxml.com/forum/post28651.html#p28656 Adrian suggested the following:

"If you're only interested in the file path indexing (quick), you can simply disable the content indexing (slow): Options > Preferences, Open/Find Resource, Enable searching in content."

Unfortunately, that does not seem to work. Even if I disable searching in content, the indexing takes more than 10 hours and after finishing, I am able to search in the content.

Our setting:
- we use a WebDAV resource (eXist server 2.0)
- created an oxygen project file (xpr) of this WebDAV resource
- disabled the option "Enable searching in content"
- started "Open/Find Resources ..."
- indexing begins automatically, but seems to ignore the disabled option "Enable searching in content"

This occurs with Oxygen 15.1 (Windows) as well as with Oxygen 14.2 (Windows).

Best regards
Jörg

[adrian]

Hello Jörg,

There seems to be a bug in the way the options are handled in v15.1. If the remote resource content indexing is enabled, but the general content indexing is disabled, the content of remote resources is indexed anyway (even though you can't actually search it).

Go to Options > Preferences, Open/Find Resource and check if the option "Index the content of remote resources" is enabled (even if it's greyed out).
If it's enabled, you'll have to disable it. To do this, you'll have to temporarily set the option "Enable searching in content" so that the other options can be accessed, then after disabling the other option ("Index the content of remote resources"), you can clear it again.

I've submitted this to our issue tracking tool to be resolved in the next release of Oxygen.

Regards,
Adrian

[joerg.hambuch]

Hi Adrian,

thanks for your reply. Unfortunately the option "Index the content of remote resources" was already disabled. I enabled it now, pressed "Apply", disabled it again, pressed "Apply" and "Ok". But unfortunately the problem persists, still the indexing is quite slow.

Regards,
Jörg

[adrian]

Hi,

There shouldn't be any indexing (not even paths) for remote resources, if that option is disabled. We've tested various scenarios, but could not reproduce the problem with that option disabled.

Maybe there's something we've missed. Could you please send us your Oxygen project file (.xpr) for analysis?
Please send it to support AT oxygenxml DOT com

Regards,
Adrian

[joerg.hambuch]

Hello Adrian,

may I pick up the thread again? Within the <projectTree> tag my original xpr-File contains about 200 000 lines like this:

Code: Select all

<projectTree ...>

    ...

    <folder name="ag-ah">

        <folder name="ag-ah"/>

            <file name="http://myserverurl.net/something/items/a/ag-ah/A%27Ha.xml"/>

            <file name="http://myserverurl.net/something/items/a/ag-ah/A%27Hab.xml"/>

            <file name ...

            [... ca. 997 more ...]

    </folder>

    <folder name="ai-am">

        <folder name="ai-am"/>

            <file name ...

            ...

    </folder>

...

There are about 200 folders, in each there are about 1000 files.

Unfortunately I am not allowed to reveal our data, so I have to remove parts of the paths and filenames. I plan to remove most of the lines containing paths and filenames and leave just a few, so you can understand the scheme. Is that OK for you?

Thanks a lot in advance,
Jörg

[adrian]

Hello Jörg,

To be honest, we haven't tested with that many remote resources linked in the project.
200000 seems a bit excessive.
I've submtted a request for our QA team to test this scenario.

Regards,
Adrian

[joerg.hambuch]

Hello Adrian,

did your colleagues from QA get any insights from testing? Here the problem still occurs.

Best regards,
Jörg

[adrian]

Hi,

I'm sorry. We have tested this, but I forgot to write back to you.

When indexing the file names/URLs for file links, the indexing time is proportional to the number of links, though it's not as large as what you described. We have tested with 20k links and it takes about a minute to index and for 100k links it takes around 6 minutes. The content of the files is not indexed and does not affect the time spent on this.
For 200k it should take around 12 minutes. This is still unacceptable, but nowhere near the 10 hours you mentioned.
This problem does not affect indexing of linked local folders. For 20k files in a linked local folder it spends 3-4 seconds.

An issue has been submitted on our bug tracking tool for this problem and we'll try to optimize the file name/URL indexing for file links.

Regards,
Adrian

[joerg.hambuch]

Hi Adrian,

thanks for your quick reply.

You mentioned before, that oXygen's search functionality changed from oXygen XML Author Version 14.1 to 14.2. When I use oXygen 14.1 on my Mac to index a oXygen .xpr file with 800000 files (which are stored in an eXist DB and accessed via eXist's build-in WebDAV), it is done within seconds.

Is there no possibility to use the same functionality in Version 15?

Best regards
Jörg

[adrian]

Hi,

Unfortunately you can't use the old project indexing in v15, it has been replaced with the Lucene indexing (specialized in content indexing). That's why you can't reuse the old index files as you wanted to on the other forum topic.

Regards,
Adrian

[joerg.hambuch]

Dear Adrian,

you wrote

An issue has been submitted on our bug tracking tool for this problem and we'll try to optimize the file name/URL indexing for file links.

May I ask if there are any news?

Best Regards
Jörg

[adrian]

Hi,

The indexing of file links has been optimized in the current development stream.
This optimization will be included in the v16.0 release (to be released very soon).

Regards,
Adrian

[adrian]

Hi,

v16.0 has been released and includes the optimizations for this situation.

Regards,
Adrian