dev: discourse findings/suggestions

hey @sb56637 are you hosting your own mailserver or using paid service? figuring out email is next on my list. i guess using sendmail isn’t an option for me anymore lol

clear history/cache/cookies, fresh login. in sidebar, collapse all sections except “my threads.” hide sidebar. unhide sidebar. blf-specific links (rules, sellers, admin) appear in “my threads” section.

1 Thank

Got it, thanks! That seems like a sufficiently obscure edge case so as to leave it be for now until a better upstream Discourse solution comes down the pike. But if it’s really getting in the way of frequent users then I could revert to the old code that also implemented “My Threads” with custom JS. (And if you thought the current code block was ugly just wait until you see what it requires for customizing two menu sections… :wink: )

Yep, no more sendmail . :slight_smile: I’m self-hosting. I’m happy with how it’s working now, but it was definitely the source of massive headaches during the first 10 days or so after the migration. It wasn’t due to the actual technical performance of the mail engine, but rather the stupidity of Yahoo / AOL and ATT / Verizon / SBCglobal email. I was totally broadsided by them summarily rejecting all emails from the new server despite it passing multiple different email deliverability tests with flying colors, whereas they were handling emails just fine from the old server that didn’t even have DMARC or DKIM implemented. They apparently have some circuit breaker style dumb rate limits that triggered from the increase in BLF mail volume with the password reset emails, and possibly the new IP address and DNS changes too, despite everything being perfectly legit. So it basically boiled down to going through the Yahoo and AT&T processes to personally contact the postmaster and get the block manually removed. These headaches might not represent the experience that all self-hosters would have; it could be limited to relatively large forums that are migrated to a new IP range and require a password reset for all users. And I suspect that I would have had similar problems with Yahoo and AT&T (and all of the other email domains they handle) if I had been using a commercial mail sender too, since the problem wasn’t with industry standards compliance but rather dumb rate limits.

It’s definitely not getting in my way. I just thought I should mention it in case you hadn’t already noticed it. Something to keep in mind for later, during a future site update. It’s a bug, but a very low-priority one which can easily wait until there’s a better solution available from upstream.

I’ve been self-hosting email for a long time, but I’m also pretty overdue for some upgrades. One of these months I’ll get around to it, but I’m really not looking forward to that 10 days of headaches. Granted, I’ll have different headaches than you did, because I have different needs. But still. It’s one more week worth of infrastructure projects I need to do.

1 Thank

any mailserver pointers? a good guide you found somewhere perhaps?

hm, i just had a chance to read your js.

you can fix it with a more specific selector like .querySelector('#sidebar-section-content-community') instead of .getElementsByClassName("sidebar-section-content")[0].

also, i would rewrite it to use MutationObserver. it’s perfect for this. it’s the current preferred way to watch for elements to appear/change. i use it in my userscripts. your code is basically a userscript :slight_smile:

2 Thanks

Yes, probably. As far as I can tell, the sidebar selectors changed during an update a week ago, so the specific one it uses now may not have been available when sb wrote the script. I noticed because my local sidebar overrides broke overnight and I had to rewrite them.

A good change overall though… The new selectors seem a bit nicer than the old ones were.

1 Thank

Nice! Thanks a ton, seems to fix that particular issue.

Right. My custom menus were also partially broken with respect to the custom order I had established, which hastened the adoption of the Discourse custom sidebar method.

1 Thank


well, when an id is not available for matching, a couple of example approaches more specific than positional selection:

// css leveraging `data-section-name` attr
document.querySelector('[data-section-name="community"] .sidebar-section-content');

// or, worst case scenario, an approach like this is always possible:
// xpath for a .sidebar-section-content sibling following a .sidebar-row containing text 'Community'
document.evaluate('//*[contains(@class,"sidebar-row") and contains(.,"Community")]/following-sibling::*[@class="sidebar-section-content"]', document).iterateNext();

just some ideas for future userscripting.

1 Thank

To follow up on this,

Turns out this is for external FAQ pages. (Very weird option.)

I tried switching the content to make /tos contain the BLF Rules and /faq be the Commercial Seller Rules, URLs to but it turns out that there is a hard-coded link from /rules/faq, and I already have a lot of public posts linking to /rules with the intention of it leading to the BLF Rules.

So I switched it all back, but there was a small improvement gleaned:

This definitely should point to the BLF Rules. So I found a way to edit the string translation to include a direct URL pointing where I need instead of a placeholder:

By registering, you agree to the <a href="%{privacy_link}">privacy policy</a> and <a href="/rules">terms of service</a>.

yeah. it can be used that way, but it doesn’t have to be. i have it linking to a topic on the forum. also note there are SiteSetting.tos_url and SiteSetting.privacy_policy_url that can be used to link external resources too.

however, the SiteSetting.faq_url is specifically goofy (yet useful) because it also splits off a route from the guidelines page and activates stuff in the /about template lol so you get an extra section there (5 instead of 4).

it wasn’t documented anywhere to my satisfaction, so i looked at the code and experimented and then documented it here.

if you like, give that a careful read and then my post with specific suggestions will probably make more sense.

not when you use SiteSetting.faq_url :wink:

the end result of my suggestion would work like this.

/guidelines, /rules, /conduct → Commercial Sellers rules

/faqyour FAQ

/tos → BLF Rules (& any extra ToS you may have) – this is automatically featured in the appropriate points in the user registration process

/privacy → privacy policy (and probably featured in user registration)

yeah, that is a problem. aside from the problem of having to update your pages, the distinction between /rules and /tos to mean commercial sellers rules vs general rules is not clear if you care about meaning in urls. that said, the same problem exists with the current setup as well.

updating the public facing posts to reflect the new url is annoying but not insurmountable. you could use the site search to find them, or for more precision, you could use a rails active record query (even using SQL if you want).

you could even use such an approach to automatically fix them all in one go. i could help if you want.

or a completely new option could be to create your own url routes and change the /about template, but i have not explored the plugin system yet, and i don’t know what plugins are capable of yet. and i’d only want to do that with a plugin because i wouldn’t want to make my own patching system for a feature of such low importance.

ps: i’m rewriting the sidebar injector to use mutationobserver because i want it for my site. will share it here when i’m done.

1 Thank

The thing is I don’t really care about having the BLF Discourse FAQs page on the 3 or 4 buttons after /about. Very few users would ever find it or think to look for it there unless I added it to the Community menu (and not hidden under More…). But I guess that setting faq url to /t/new-blf-forum-engine-faqs/217457 kind of works because that way /rules still points to the BLF Rules, which is important for me to maintain because that’s what has the most historical links to it. I’ve had more than enough experience with regexp to know that I really really don’t want to deal with replacing strings across millions of posts. A heads-up for you about that:

  • I heavily tested the rake posts:remap text replacement functionality, and I found it to be totally random about replacing a fixed number of strings followed by a small seemingly random number of strings, and it never replaced all of them.
  • I also extensively tried using the Postgres console directly with the regexp_replace function, but it was also terribly unpredictable because Postgres doesn’t support the most common Perl-Compatible Regular Expressions (PCRE) flavor that I tested my regexp with.
  • Also editing the raw post content would require rebaking all the posts, which would mean I would have to take the forum offline for a few minutes to rescale the VM to have much more RAM and CPU cores, and then let the rake posts:rebake process crunch away on millions of posts for about 48 hours. [2023-02-14] Apologies for the downtime was the result of trying to rebake all the posts with the current VM while online. After that I performed many troubleshooting attempts with a snapshot of the system in another VM, and I even tested different flavours of Linux with all sorts of combinations of swap files and/or zram, but it could not complete the rebake due to memory problems and bizarre CPU hang issues, producing fatal errors ranging all the way from Ruby errors to kernel panics. And that was without any live traffic.

So I can live with this I think:

Nice! Thanks! Looking forward to trying it.

this is the right way of doing it :sunglasses:

it uses promises and MutationObserver.

absent a bug, it should be free of jank. any kind of collapsing, expanding, and refreshing should just “do it” in a performant way.

that took me way too long to write. i’m not a js guy. give it a proofread. maybe i missed a semicolon or something somewhere.

when the sidebar mutates, a promise is created for each link that is resolved when the link’s relevant section is available.

specify injections by section. kebab must be text in kebab case. you also specify which existing element to inject before or after with its kebab’d suffix (sidebar-section-link-SUFFIX), which should just be the name of the link in kebab case anyways.

you can also remove elements. just make removals an empty array if you’re not using that.

edit: i made some changes to the script and made more positioning options. just refer to the example at the top of the script. append: true is probably the positioner you want to use.

2 Thanks

Nice! Really appreciate you sharing that. I’ll be pretty busy these next couple of days, but I hope to give your code a try some time this week.

also there is some logging in there. maybe leave it while playing with it and testing, but you can remove those few console.log lines after.

1 Thank

i actually developed it using tampermonkey (browser extension to inject userscripts into pages), but i haven’t succeeded in getting it to work yet from the discourse admin interface as a theme component yet. hrmmm

will update when i get it working


oh, i see what the problem is. i developed it using tampermonkey + your site.

my code only fires upon sidebar mutation. i never handled the initial case where the sidebar is present, but no mutations have happened (like when you refresh with the sidebar open).

my code was working without that because your existing sidebar code was mutating the sidebar. lol

1 Thank

ok all fixed.

to test, just paste the whole thing into the head section of a new theme component. append: true is probably the only positioner you need, but i made it pretty flexible.

1 Thank

fwiw, i thought your faq was very helpful, as BLF (post-changeover) was my first in-depth exposure to discourse. i also based my faq on yours.

when i change over, i’ll actually have my FAQ page (a topic) pinned as the banner topic for a month or so with details about the migration and how to get the most out of the new forum. i’ll also leave a FAQ link in the sidebar.

on BLF, it’s kinda had to find though since it’s not in the sidebar, and /about (which prominently links to it now) isn’t in the sidebar, either. i think it doesn’t hurt to prominently display the instruction manual, especially when it’s a nice document :slight_smile:

1 Thank