Page 1 of 1

Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Aug 16, 2023 5:58 am
by galanohan
Hi,

To solve the search limitation on CJK keywords with more than 3 characters, as Alin suggested, I've deployed the trial version of oxygen feedback cloud for the webhelp published here: https://cdn.dolphindb.cn/help/index.html

In general, the feedback system looks great. I'm totally satisfied with its backend design, and frontend layout. It's easy to deploy and manage, and also quite handy for readers to put comments to page or block they have questions with.

However, the root cause of deploying a trial version of oxygen feedback system was to solve the search limitation on CJK characters in Webhelp. I've been struggling in oxygen xml editor and config files for weeks, also it might be a solution to put keyword/inderterm tags to every topic, but thinking about the total amount of files exceeds 2000, I believe that's a painful journey to us all ... Then I deployed oxygen feedback cloud.

In the console, I enabled the content indexing and had the content indexed for several times:
image.png
image.png (24.68 KiB) Viewed 2940 times
But still, in our online help, https://cdn.dolphindb.cn/help/index.html, if you pick any random chapter titles with more than 3 characters and use them as keywords, no results returned.
image.png
image.png (92.72 KiB) Viewed 2940 times
image.png
image.png (12.71 KiB) Viewed 2940 times
image.png
image.png (12.76 KiB) Viewed 2940 times
Give it a try if you can. You don't really need to understand Chinese, just pick any 3-Chinese-character string, paste them to the search bar and hit the search button.
Or maybe some of my configuration in the backend was wrong? Let me know if I need to provide any further details that could help solve the problem.

Thanks and Regards,
Galano

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 17, 2023 9:47 am
by alin
Hi Galano,

Thank you for your feedback!
We are currently investigating the cause of this issue. We will update this thread when we come to conclusion.

Regards,
Alin

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 17, 2023 11:03 am
by galanohan
Thank you!

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 17, 2023 1:07 pm
by galanohan
@Alin

In case you guys test the feature over the url I provided above, let me know if I need to turn on the content indexing in the site configuration.
I turned it off hours ago, because I found when it's on, even the English search results cannot be displayed. When it's off, at least searching English keywords works well as expected.

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Tue Aug 22, 2023 12:08 pm
by galanohan
@Alin

One question from our readers: to add comments on a page, does a guest need to register over oxygen's site? or does it depend on the license of oxygen feedback? I noticed the ultimate version of oxygen feedback has no limit on moderators.

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Aug 23, 2023 11:40 am
by alin
Oxygen Feedback requires users to be authenticated so that they can post comments. The subscription does not have any restriction for the number of commenters.

In addition to users roles that have administration and moderation rights, the following roles are available:
  • Regular User
    • has rights to post new comments
    • their comments require approval
  • Trusted - comments are automatically approved
  • Banned - cannot post comments
When a user creates their account from the Oxygen Feedback Commenting Component embedded on a website they are assigned with the Regular User Role. The new account is created in the same organization where the current site configuration was defined.

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 24, 2023 4:19 am
by galanohan
Thank you! I'll inform my colleagues who weren't granted admin roles to give it a test.

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 31, 2023 11:29 am
by alin
Hello,

We have released a new version of Oxygen Feedback (v. 3.0.3). In this version we have tried to improve the search engine to return better results for Chinese content. Please give it a try!

Regards,
Alin

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 31, 2023 1:01 pm
by galanohan
It works very well!

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Aug 31, 2023 1:12 pm
by galanohan
Another problem comes up:

If I set the content indexing language as Chinese, then function names, parameter names or keywords in SQL expression cannot be searched, as they were processed as English. Is there a way to index two languages?

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Fri Sep 01, 2023 5:44 pm
by alin
Hello,

Please send us a minimal DITA sample on which the problem can be reproduced.

Regards,
Alin

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Sat Sep 02, 2023 7:11 am
by galanohan
md_dita_zh_en.zip
(19 KiB) Downloaded 305 times
Hi Alin,

please find the sample package from attached.
image.png
image.png (4.79 KiB) Viewed 2665 times

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Sep 06, 2023 11:07 am
by Costin
Hi Galano,

Thank you for the feedback you provided and for the sample! We were able to reproduce the behavior you reported using your sample.
We are currently prepairing to release a new version of the Oxygen Feedback, so almost all of our HR are focused on implementing the features to be included in the new release, which should Occur in late October.
However, I have just logged an issue in our internal issues/improvements tracking system, altogether with the feedback and the same files you provided.
After the new version release, our development team will look into the issue and try to find a fix to be implemented in a future version of the Oxygen Feedback.
We will notify this thread when/if a new implementation will become available.

Regards,
Costin

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Sep 06, 2023 11:52 am
by galanohan
A workaround I could think of to bypass the issue I reported is:
1. write a script that split any given CJK keywords for search into multiple groups separated by blank space. For example, like this:
image.png
image.png (23.34 KiB) Viewed 2595 times
2. When the on-click event occurs after user clicking the search button, the script above is triggered.
3. The search results returned by the script in step 1 is then returned to the search result page.

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Tue Sep 12, 2023 12:31 pm
by beniamin_savu
Hi,

Indeed, it seems that with this workaround it may be possible to add the spaces required by the search engine to work.

Regarding this aspect, we managed to create a sample publishing template that changes the search query after the user clicks the search button. In this example we added the string "modified" to the end of the search query entered by the user. The zip file for this sample publishing template can be found here:
ChangeSearchQuery.zip
(195.26 KiB) Downloaded 289 times
You can use this publishing template as a starting point to modify the search query according to your needs. Please note that this will require a certain level of JavaScript expertise. To modify the functionality you will have to edit the custom.js file found at: ${pubishingTemplateFolder}/js/custom.js

Best regards,
Beniamin Savu
Oxygen WebHelp Team
http://www.oxygenxml.com

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Sep 13, 2023 7:08 am
by galanohan
Hi Benjamin,

Thanks for the custom.js. I modified line 13 to the following:

var executed = false;
$( document ).ready(function() {
$("#searchForm").on("submit", (e) => {
// triggers the submit event handler multiple times.
if(!executed) { // execute it only one time.
e.stopPropagation();
var userQuery = $('#textToSearch').val();
if (userQuery.trim() === '') {
e.preventDefault();
return false;
}

//userQuery = userQuery.replace(/[\u4e00-\u9fa5]{2}/g, '$& '); // split every given Chinese input by separating every two Chinese characters by spaces.
userQuery = userQuery.replace(/[\u4e00-\u9fa5]/g, '$& '); // split every given Chinese input by separating every Chinese character by spaces.
$('#textToSearch').val(userQuery);
executed = true;
}
});
});

As you can see,

//userQuery = userQuery.replace(/[\u4e00-\u9fa5]{2}/g, '$& '); // split every given Chinese input by separating every two Chinese characters by spaces.
userQuery = userQuery.replace(/[\u4e00-\u9fa5]/g, '$& '); // split every given Chinese input by separating every Chinese character by spaces.

Both works pretty good. But given the fact that CJK string with no more than 2 chars as search input has no problem in text search, string split by every 2 chars seems to be able to return more accurate results, especially when the keyword for search contains only 2 chars.

Thanks so much!

Btw, now the local webhelp search is solved by this custom.js, will the webhelp output conflict with the content indexing in oxygen feedback if the content indexing is based on Chinese or English?

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Wed Sep 13, 2023 1:04 pm
by cosmin_andrei
Hello,
will the webhelp output conflict with the content indexing in oxygen feedback if the content indexing is based on Chinese or English?
Could you please provide more details in order to better understand your use-case?

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Thu Sep 14, 2023 9:33 am
by galanohan
For example, with the help the preceding custom.js, search with Chinese string with more than 2 characters is now available in local Webhelp output.
Now, if I deploy the Chinese webhelp package to our company site, and then specify the directory of index.html at a certain site in oxygen feedback, enable the content indexing, and set the indexing lanuage as Chinese, and save the configuration. When the indexing completes for the content in the directory specified in oxygen feedback site, the text search in the specified URL will use the text search provided by the oxygen feedback, and text search supported by js/custom.js in the publishing template will be overridden, did I understand the login right?

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Fri Sep 15, 2023 5:43 pm
by beniamin_savu
Hi,

The custom.js file is executed regardless of how the WebHelp output is deployed, locally or on a server. Oxygen Feedback's search function will not override the behaviour of the custom.js file when used in the WebHelp output. The custom.js will still be executed and will split the words according to your changes.

We suggest that the custom.js file to be applied only if the WebHelp output is run locally.

Best regards,
Beniamin Savu
Oxygen WebHelp Team
http://www.oxygenxml.com

Re: Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

Posted: Sun Sep 17, 2023 4:36 am
by galanohan
Thanks for the advice!