Reading through the session notes and this proposal, several aspects come to mind that I believe would be worth mentioning. I'll try to give a quick brain dump:

• Distinction between public services (that clients use instead of or in addition of mediawiki), and internal services (that are used by mediawiki).
• Distinction between stateless and stateful services. MediaWiki uses several external services to maintain state (MySQL, memcached, redis, etc).
• Distinction between long-running tasks (transcoding), async notifications (event logging), and tasks to be performed synchronously during a request "while the user waits" (like the proposed new session storage service).
• Per the above, stateless services that perform synchronous in-request tasks tend to be pointless. The same functionality can generally be implemented inline, in core or as an extension, without the communication and operational overhead of a standalone service.
• Discussion of whether HTTP is the preferred method of communication between services (probably yes, at least for synchronous calls)
• The "chattyness with the API" criterion also works the other way around: if MediaWiki would have to talk to the service a lot, that's an indicator that the functionality should probablly not be in a separate process.
• Having the need to be synchronous as a counter-indicator doesn't seem useful. Several prime candidates for external services would need synchronous call semantics, e.g. an authentication service.
• Functionality invoked via shell-out or implemented by forking should be treated separately, as it has very different characteristics.
• The statement that it's "very likely" that an extension would be used for integration is misleading, as it implies that functionality that is presently in core should not be factored out into an external service. But factoring core functionality out into external services should indeed be considered, especially for security relevant functionality.
• Anything that needs to function in a shared hosting environment (LAMP) can't be an external service (or needs to have an alternative PHP implementation)
• Anything that needs to parse wikitext or need access to MediaWiki i18n messages (which are wikitext) should probably not be an external service.

That's it from the top of my head. Too bad I couldn't be in the session at TechConf.

I think @daniel raises quite a few good points, I will try to go through the document and integrate some more observations in the coming days. But I'd first propose we move the discussion mostly on-wiki: this is a big wall of text, and I feel a wiki page it's a better place to collaboratively enhance it.

Also, this is very large, and I can see, from a first read, quite a few things that are needed are missing. I would suggest we split this RfC into separate sections that we could discuss separately, and more specifically:

1. Selection criteria (as above)
2. Architectural principles for service-service interaction (so communication methods, failure management, retry policies, etc. this section is not very well defined in the current document and would need expanding
3. Architectural Implementation Guidelines (as above)

@Imarlier if you agree, I'll move this document to mediawiki.org and we can discuss how to integrate it there? Else, I'll start proposing amendments here.

@daniel one detail I'd want to add: external storage systems like mysql or redis should not be considered "external services" - they're part of where MediaWiki stores its state.

Services that are used to augment the functionality of MediaWiki in a meaningful way, have their own storage (if any) are external services.

So, just to make concrete examples:

• redis-for-sessions is not a service, as it's used as a pure datastore from Mediawiki, which is where all business logic is stored
• the new Session Storage service, if it will include some business logic on the service side (like rate-limiting, security checks, invalidation logic), should be considered an external service.

We will need guidelines for data storage adoption, sure, but those should be in a separate document.

@Joe the only distinction I see there is "stuff we write" vs "stuff we don't write", really.

But you are right that criteria for data storage technology selection don't belong here. But "we use an external service for storing X" should follow from this guideline, even if that external service is Redis or Swift. But the criteria for deciding *which* data storage tech to use do not belong here.

Actually, Swift over local files is a good example for a case where "inline" functionality was replaced with an off-the-shelf service. And the criteria developed here should apply to, and be consistent with, that decision.

In T208524#4731479, @daniel wrote:

@Joe the only distinction I see there is "stuff we write" vs "stuff we don't write", really.

But you are right that criteria for data storage technology selection don't belong here. But "we use an external service for storing X" should follow from this guideline, even if that external service is Redis or Swift. But the criteria for deciding *which* data storage tech to use do not belong here.

Actually, Swift over local files is a good example for a case where "inline" functionality was replaced with an off-the-shelf service. And the criteria developed here should apply to, and be consistent with, that decision.

Ok, I don't think the same rules should apply for storage backends and business logic services - the main driver for having external storage backends is usually something absolutely tangential to the criteria expressed here - either scalability, functionality or storage guarantees.

In short:

• We already have guidelines for choosing if local storage or a storage backend is preferrable. I think those can be summarized with "it is absolutely forbidden to write something as if the local disk was available for non-ephemeral storage", where non-ephemeral means "expects such state to be preserved across requests".
• The quality criteria (about instrumentation, ownership, stability, language choice, architecture) we must apply to a storage backend are quite different than the ones you'd apply to an actual application performing business logic.

Broadening the scope of the principles expressed here would weaken the prescriptive value of this document (and - I guess - the more detailed implementation documents we will have to write starting from this.

Since I got no feedback on the proposal to move this on-wiki, I'll start commenting and amending here, although I would eventually like to move everything on-wiki.

Section 1: definitions

I don't think it's correct to define "external service" any executable that we launch shelling out from MediaWiki. For the sake of simplicity, I'd go further and assume "external service" means a separate logical cluster that MediaWiki reaches via an IP connection.

Section 2: selection criteria
This is the section I find more problematic, as basically none of our currently deployed services would fit that description.

I would remove "Feature is either async to servicing requests, or if sync, provides optional features." from the "DO's" and " Feature is synchronous to the request, and the request cannot complete until the feature is successfully provided." from the DON'Ts because they're worded in a misleading way. What we really want, if we're writing a sound architecture, is that the failure of an external service will not compromise the functionality of the rest of the "application" as seen from the user.
To make an example using CirrusSearch, which was cited here:
If the elasticsearch cluster is down or very slow, the search extension in mediawiki will just time out with a sensible default, and not block the functionality of the rest of the wikis. Search will be unavailable but the sites will be nonetheless up and running.
That's ok. What we need to avoid is to make daisy chains of microservices (so that service A cannot work unless service B works too - with a possible exception, in our case, for MediaWiki itself, that might be fundamental for the functionality of other services).

I am also definitely not ok with "A desire for the feature to be owned and maintained autonomously, pace of development is unlikely to correspond to the pace of MediaWiki development, or other organizational factors" as a criteria for having an external service.
This statement is unquantifiable and sounds like a "free-for-all" card that can always assumed to be true by anyone, and hard to prove or disprove. Moreover, it's the only criteria amongst all the ones we established that is not technical in nature. I have a further reason to find this problematic: this principle was used is the justification that was put forward in the past for creating external services (some of which are now without an owner following reorgs/strategy shifts) when there was no real need for it.

I think we should instead indicate clearly that, architecturally, creating an external service adds complexity to the infrastructure, and thus should be done only if there are strong reasons for building it rather than just a MediaWiki extension.

@Imarlier you wrote this RFC as the outcome of a session that you took on at the last minute. I suppose it's not fair to expect you to go through with getting this RFC through the process. Reaching consensus on this is not going to be quick or easy. Are you interested in (and have capacity to) take this on? If not, we should designate someone else to drive this.

If no one else wants to pick this up, I'm willing to work on this RfC. The fact I didn't participate in the session can be both and advantage (I have a fresh prespective) and a disadvantage (I don't have more background than what's written here).

Can we add to the "and in addition must" criteria, something along the lines of, have someone responsible for fixing any security issues that come up. Particularly for things we make ourselves, they don't just need to pass security review now, there also needs to be someone responsible for responding to security issues over the long-term, possibly long after development is done. (Things like the lack of response on T207222 are making me concerned about this point)

@daniel @Joe I don't think that I'm really the right person to drive this RfC -- I took on moderating the session, and I think that it's important that there are architectural guidelines in place around the integration of external services, but I don't pretend to be qualified to represent any particular community consensus on this question. I'm happy to continue to be involved to the extent necessary/desired, but don't have any attachment if someone else wants to pick it up.

(I will note, though, that someone should be driving this. I do think that there's a consensus, or close to it, that having actionable guidelines for external vs. internal decisions is important in the very near term.)

@Joe on-wiki is fine, from my perspective.

@Joe I think it would be excellent if you could take this on, thank you!

+1 to moving on-wiki.

I'll also throw in a quibble with this criterion: "A non-PHP language or framework exists that significantly simplifies implementation." To me it seems better to keep the focus here squarely on architectural principles, and to consider the language out of scope as an implementation detail (subject to criteria set elsewhere about things like how many languages we can practically support in production). For our purposes here, could we simply say, "An external tool or framework exists that significantly simplifies implementation"?

In T208524#4744197, @Mholloway wrote:

+1 to moving on-wiki.

I'll also throw in a quibble with this criterion: "A non-PHP language or framework exists that significantly simplifies implementation." To me it seems better to keep the focus here squarely on architectural principles, and to consider the language out of scope as an implementation detail (subject to criteria set elsewhere about things like how many languages we can practically support in production). For our purposes here, could we simply say, "An external tool or framework exists that significantly simplifies implementation"?

I agree we can be more generic in wording ("implementing the feature in a different language or framework significantly simplifies implementation") but the concept as epxressed is correct - and a good reason for developing a functionality as an external feature.

I will start moving the RfC on-wiki, and add some proposed amendments in the talk page.

I think the current version of the RfC is reasonably well structured, to the point I think we should move the discussion here.

In T208524#4731042, @daniel wrote:

• Anything that needs to function in a shared hosting environment (LAMP) can't be an external service (or needs to have an alternative PHP implementation)

I strongly second this point, and I see it's missing from the current version of the RFC. One of MediaWiki's strengths as a software product outside of Wikimedia is that it generally works for hosting a low-traffic wiki without a whole lot of MediaWiki-specific setup and configuration.

For example, on a LAMP stack MediaWiki will use a MySQL table for search and key-value storage, while more specialized installations can use Elasticsearch (via the CirrusSearch extension) for the former and memcached, redis, and/or others for the latter. The job queue on a generic LAMP stack is run opportunistically on requests post-send, while more specialized installations can run jobs via cron or do something like Wikimedia's job runner architecture. While not a service, Scribunto on a LAMP stack will shell out to a standalone Lua binary while more specialized installations can install the LuaSandbox PHP extension, and we do similar things in core for image scaling and diff generation. Personally I've always thought it a bit of a failing that WYSIWYG editing (VisualEditor) only works with Parsoid and Restbase, and I'm hopeful that Parsoid-PHP will fix that.

There's also a bit of a spectrum here. Services that have their own communities (memcached, redis, Elasticsearch, etc.) and are packaged by most Linux distributions are likely easier to set up for third parties than bespoke services designed for use in Wikimedia's infrastructure, if only because externally-developed services' communities have had more people installing them in more different environments.

In T208524#4894412, @Anomie wrote:

In T208524#4731042, @daniel wrote:

• Anything that needs to function in a shared hosting environment (LAMP) can't be an external service (or needs to have an alternative PHP implementation)

I strongly second this point, and I see it's missing from the current version of the RFC. One of MediaWiki's strengths as a software product outside of Wikimedia is that it generally works for hosting a low-traffic wiki without a whole lot of MediaWiki-specific setup and configuration.

That's not really a requirement for external services, and while I tend to agree that MediaWiki needs to be an autoconsistent standalone software, I'm not convinced there is consensus on this point, or on the details of it.

Actually, I think the wording used here is also completely wrong. One could have an external service written in PHP and host it in a LAMP environment under a separate URL and work like a charm.

That MediaWiki keeps working correctly in the future when you just clone its repository (or download the tarball) is a requirement of MediaWiki, not of the Wikimedia architecture.

For example, on a LAMP stack MediaWiki will use a MySQL table for search and key-value storage, while more specialized installations can use Elasticsearch (via the CirrusSearch extension) for the former and memcached, redis, and/or others for the latter. The job queue on a generic LAMP stack is run opportunistically on requests post-send, while more specialized installations can run jobs via cron or do something like Wikimedia's job runner architecture. While not a service, Scribunto on a LAMP stack will shell out to a standalone Lua binary while more specialized installations can install the LuaSandbox PHP extension, and we do similar things in core for image scaling and diff generation. Personally I've always thought it a bit of a failing that WYSIWYG editing (VisualEditor) only works with Parsoid and Restbase, and I'm hopeful that Parsoid-PHP will fix that.

Gracefully degrading a feature from a full-fledged service to a simple implementation in the LAMP stack makes sense for features that are fundamental to mediawiki (async processing, search) but IMHO not in every case. While I agree that's desirable to have VE available to most installations, I don't think that's the reason why implementing parsoid outside of MediaWiki was a bad idea.

There's also a bit of a spectrum here. Services that have their own communities (memcached, redis, Elasticsearch, etc.) and are packaged by most Linux distributions are likely easier to set up for third parties than bespoke services designed for use in Wikimedia's infrastructure, if only because externally-developed services' communities have had more people installing them in more different environments.

I don't consider storage backends as "external services", and this RfC is most surely not directed at those. This is a set of standards for when and how to implement a feature within our infrastructure.

So while I understand the concern you and @daniel have, I'm not sure how that requirement fits this RfC rather than a set of principles regarding how we develop MediaWiki (not delegating completely core functionalities outside of the "monolith").

I'd like to clarify on the point of "Anything that needs to function in a shared hosting environment (LAMP) can't be an external service (or needs to have an alternative PHP implementation)"

If there is a feature that we want to implement outside of MW core, as a standalone service, not in PHP, there are then three options:

1. we make explicit that this feature will simply not work in a minimal environment (that is, the feature is optional).
2. we provide and maintain an alternative in-process implementation in PHP.
3. we drop the requirement that "minimal environment" means "LAMP stack with no root access", and require e.g. container based hosting instead.

Parsoid went with (1). The problems we have with it are mainly caused by its need to call back to the core API all the time. As Anomie points out, we use (2) for search. For number (3), I think the question is when and how, rather than if. One, three, five years?

The reason I think this point should be mentioned in the standards is that these three options need to be considered when making the decision on implementing some functionality as an external service. For every such service, we need to explicitly choose one of the three options, and document the rational behind that choice. I agree that this is dictated by requirements of MediaWiki rather than Wikimedia architecture, but requirements of MediaWiki are constraints placed on the Wikimedia architecture, so they need to be considered.

In T208524#4895738, @Joe wrote:

Actually, I think the wording used here is also completely wrong. One could have an external service written in PHP and host it in a LAMP environment under a separate URL and work like a charm.

I suppose you could. Although that would mean you passed up the opportunity to make your service a library installable with Composer so it could be pulled into MediaWiki via composer.json and used that way without the overhead of API calls to localhost.

That MediaWiki keeps working correctly in the future when you just clone its repository (or download the tarball) is a requirement of MediaWiki, not of the Wikimedia architecture.

That strikes me as rather short-sighted. As Daniel points out, constraints on MediaWiki are also constraints on Wikimedia architecture, unless we plan on abandoning MediaWiki for use outside of Wikimedia's infrastructure.

Gracefully degrading a feature from a full-fledged service to a simple implementation in the LAMP stack makes sense for features that are fundamental to mediawiki (async processing, search) but IMHO not in every case. While I agree that's desirable to have VE available to most installations, I don't think that's the reason why implementing parsoid outside of MediaWiki was a bad idea.

In general I agree with your first sentence, not everything needs to be a core feature available on minimal installations. But over the years WYSIWYG editing has been touted as something necessary to running an effective wiki for editing by the general public, to the point that some people were pushing for storing Parsoid-HTML as the canonical representation of the page rather than wikitext.

So while I understand the concern you and @daniel have, I'm not sure how that requirement fits this RfC rather than a set of principles regarding how we develop MediaWiki (not delegating completely core functionalities outside of the "monolith").

The RFC is titled "Standards for external services that integrate with MediaWiki". If you drop the "integrate with MediaWiki" bit, what exactly are we trying to discuss here? And why does https://www.mediawiki.org/wiki/Requests_for_comment/Standards_for_external_services#Selection_Criteria:_Deciding_whether_an_external_service_is_appropriate exist?

In T208524#4895821, @daniel wrote:

For number (3), I think the question is when and how, rather than if. One, three, five years?

I would ask "if". What benefits does doing that bring that outweigh breaking compatibility with low-traffic low-effort installations?

The reason I think this point should be mentioned in the standards is that these three options need to be considered when making the decision on implementing some functionality as an external service. For every such service, we need to explicitly choose one of the three options, and document the rational behind that choice. I agree that this is dictated by requirements of MediaWiki rather than Wikimedia architecture, but requirements of MediaWiki are constraints placed on the Wikimedia architecture, so they need to be considered.

+1 to that.

I would say that what we need is a definition of what functionality is and is not part of a "simple mediawiki installation that should work in a LAMP environment" and thus cannot opt for the third option you your list. I'd expect it's a much needed work that should come out of product people, probably. We can add a couple sentences to further clarify the relationship between MediaWiki and external services in terms of functionality.

In T208524#4896817, @Anomie wrote:

The RFC is titled "Standards for external services that integrate with MediaWiki". If you drop the "integrate with MediaWiki" bit, what exactly are we trying to discuss here? And why does https://www.mediawiki.org/wiki/Requests_for_comment/Standards_for_external_services#Selection_Criteria:_Deciding_whether_an_external_service_is_appropriate exist?

You might notice I intentionally dropped the last part from the title on-wiki, and I forgot to do the same here.

These standards should be held up for anything that we want to run in our environment.

That section you cite is there because sometimes we've implemented features in a service when it would've made sense to implement them in MediaWiki , or vice versa, from an architectural point of view.

As I said, adding a caveat (and a reference to some list of product requirements that should exist but doesn't at the moment) about the duplicating efforts that are needed if replacing a core feature with an external service doesn't do harm, and I'll do it.

In T208524#4895821, @daniel wrote:

For number (3), I think the question is when and how, rather than if. One, three, five years?

I would ask "if". What benefits does doing that bring that outweigh breaking compatibility with low-traffic low-effort installations?

The reason I think this point should be mentioned in the standards is that these three options need to be considered when making the decision on implementing some functionality as an external service. For every such service, we need to explicitly choose one of the three options, and document the rational behind that choice. I agree that this is dictated by requirements of MediaWiki rather than Wikimedia architecture, but requirements of MediaWiki are constraints placed on the Wikimedia architecture, so they need to be considered.

+1 to that.

In T208524#4896817, @Anomie wrote:

! In T208524#4895821, @daniel wrote:
For number (3), I think the question is when and how, rather than if. One, three, five years?

I would ask "if". What benefits does doing that bring that outweigh breaking compatibility with low-traffic low-effort installations?

This seems to me like a false dilemma. I can readily imagine a future where both of these are true:

• Mediawiki provides a 'core' feature set that is implemented directly in PHP, installable on a LAMP stack in the usual way, intended for low-traffic installations
• Mediawiki is also distributed as itself + a small suite of external services, with the deployment managed by docker-compose, or a set of Helm charts for deployment on k8s, or similar. This would save substantial effort for high-traffic users, or for those seeking to replicate a maximum-feature-set install.

I would ask "if". What benefits does doing that bring that outweigh breaking compatibility with low-traffic low-effort installations?

I'm happy to discuss this, but that is beyond the scope of this ticket. Investigating this question more deeply, and writing up the trade-offs involved, is on my list for this quarter. For now, the minimal platform is LAMP-without-root-access, and bare-bone MediaWiki has to work on it.

I would say that what we need is a definition of what functionality is and is not part of a "simple mediawiki installation that should work in a LAMP environment"

While more explicit guidance of this would be useful, it seems like the question whether or not (or to what degree) a given feature needs to be supported in a minimal environment has to be made on a case-by-case basis. Having a check-list of criteria to guide that decision would be great, but I don't think we need to wait for that to exist.

I added the following paragraph to the RfC:

Given MediaWiki is not just part of the Wikimedia production infrastructure but also a software used my many third-parties, we can't just delegate some of its fundamental functions to an external service completely. Whenever an external service provides a functionality, we also need to ask ourselves if said functionality is fundamental or optional to MediaWiki.If the functionality replaced by the external service is fundamental, a fallback solution must be present within MediaWiki that substitutes what is being implemented in the service. As an example: for Wikimedia purposes, async processing for MediaWiki is implemented via a series of external services; for simple installations, a mysql-backed version of the same mecahnism exists and cannot be dropped from MediaWiki. What constitutes a core functionality of MediaWiki and what doesn't will need to be further defined elsewhere, and is beyond the scope of this document.

@Anomie @daniel do you think it would address your concerns and clarify the requirements a bit?

@Joe sgtm, thanks

In T208524#4896987, @Joe wrote:

I added the following paragraph to the RfC:

Given MediaWiki is not just part of the Wikimedia production infrastructure but also a software used my many third-parties, we can't just delegate some of its fundamental functions to an external service completely. Whenever an external service provides a functionality, we also need to ask ourselves if said functionality is fundamental or optional to MediaWiki.If the functionality replaced by the external service is fundamental, a fallback solution must be present within MediaWiki that substitutes what is being implemented in the service. As an example: for Wikimedia purposes, async processing for MediaWiki is implemented via a series of external services; for simple installations, a mysql-backed version of the same mecahnism exists and cannot be dropped from MediaWiki. What constitutes a core functionality of MediaWiki and what doesn't will need to be further defined elsewhere, and is beyond the scope of this document.

@Anomie @daniel do you think it would address your concerns and clarify the requirements a bit?

Sounds good to me, thanks.

@Joe In the TechCom meeting today I agreed to collab with you to summarise RFC in the task here with a problem statement and proposed solution.

TechCom is placing this on Last Call ending March 6 1pm PST(21:00 UTC, 22:00 CET)

Would it be possible to clarify the wording on "There is no existing FLOSS software that provides the same functionality"? I believe the intent here is about surveying the FLOSS ecosystem for well crafted, well maintained, architecturally compatible FLOSS software that provides comparable functionality before specifying and building new non-trivial standalone services.

I think that the

"Collect RED metrics; be able to export those metrics according to WMF standards specified in the implementation guidelines"

should be rephrased to be more generic, e.g.

"Be able to collect and expose operational metrics according to the current WMF standards specified in the implementation guidelines"

and then specify in the implementation guidelines that we want RED metrics, or their larger counterpart, the 4 golden signals[1] or perhaps radically different approaches.

[1] https://landing.google.com/sre/sre-book/chapters/monitoring-distributed-systems/#xref_monitoring_golden-signals

"Log all requests received via the production logging facilities"

Should we make this a bit more generic? e.g.

"Log actions via the production logging facilities"

I am also a bit skeptical about this:

"Be able to perform requests via TLS to a specific hostname/ip provided via configuration"

In case it isn't clear from the bolding, it's the "via TLS" part I am talking about. The rest of the sentence looks fine to me.

Not that I don't want services to be able to do TLS, but TLS is a thing that has been proven hard to do correctly[1], [2]. It's something that perhaps we should invest in the infrastructure to do correctly and not burden service developers with.

[1] https://github.com/nodejs/node/issues/4175
[2] https://maulwuff.de/research/ssl-debugging.html#hdr3

In T208524#4982558, @akosiaris wrote:

"Log all requests received via the production logging facilities"

Should we make this a bit more generic? e.g.

"Log actions via the production logging facilities"

Agreed, will amend.

I am also a bit skeptical about this:

"Be able to perform requests via TLS to a specific hostname/ip provided via configuration"

In case it isn't clear from the bolding, it's the "via TLS" part I am talking about. The rest of the sentence looks fine to me.

Not that I don't want services to be able to do TLS, but TLS is a thing that has been proven hard to do correctly[1], [2]. It's something that perhaps we should invest in the infrastructure to do correctly and not burden service developers with.

[1] https://github.com/nodejs/node/issues/4175
[2] https://maulwuff.de/research/ssl-debugging.html#hdr3

To be pefectly clear, I don't think this, or circuit-breaking, will be implemented necessarily inside the service. The infrastructure will provide middlewares to do it, but services will need to have interfaces compatible with such infrastructural middleware, or implement the functionality themselves.

Take as an example Kask, the new cassandra-backed k-v storage service: we might not want a middleware mediating calls there. As the service doesn't need to do any circuit-breaking as it doesn't do any RPC, and it can easly expose TLS natively. So it would fulfill the requirements without the need of infrastructural helpers.

Does this address your concern?

In T208524#4982537, @akosiaris wrote:

I think that the

"Collect RED metrics; be able to export those metrics according to WMF standards specified in the implementation guidelines"

should be rephrased to be more generic, e.g.

"Be able to collect and expose operational metrics according to the current WMF standards specified in the implementation guidelines"

and then specify in the implementation guidelines that we want RED metrics, or their larger counterpart, the 4 golden signals[1] or perhaps radically different approaches.

[1] https://landing.google.com/sre/sre-book/chapters/monitoring-distributed-systems/#xref_monitoring_golden-signals

Sure, this was a case where I didn't make the requirement generic enough (given we're delegating this to implementation guidelines).

In T208524#4982033, @dr0ptp4kt wrote:

Would it be possible to clarify the wording on "There is no existing FLOSS software that provides the same functionality"? I believe the intent here is about surveying the FLOSS ecosystem for well crafted, well maintained, architecturally compatible FLOSS software that provides comparable functionality before specifying and building new non-trivial standalone services.

Sure, I'll repurpose what you wrote verbatim - it's much clearer that way!

Production deployment
<snip>
Have backups if the service stores any data

May be we could add "and a restoration/emergency plan", in terms that sometimes it might not be straightforward of what to do with a backup or what is included in a backup.

In T208524#4987369, @Joe wrote:

I am also a bit skeptical about this:

"Be able to perform requests via TLS to a specific hostname/ip provided via configuration"

In case it isn't clear from the bolding, it's the "via TLS" part I am talking about. The rest of the sentence looks fine to me.

Not that I don't want services to be able to do TLS, but TLS is a thing that has been proven hard to do correctly[1], [2]. It's something that perhaps we should invest in the infrastructure to do correctly and not burden service developers with.

[1] https://github.com/nodejs/node/issues/4175
[2] https://maulwuff.de/research/ssl-debugging.html#hdr3

To be pefectly clear, I don't think this, or circuit-breaking, will be implemented necessarily inside the service. The infrastructure will provide middlewares to do it, but services will need to have interfaces compatible with such infrastructural middleware, or implement the functionality themselves.

Yes, agreed on that.

Take as an example Kask, the new cassandra-backed k-v storage service: we might not want a middleware mediating calls there. As the service doesn't need to do any circuit-breaking as it doesn't do any RPC, and it can easly expose TLS natively. So it would fulfill the requirements without the need of infrastructural helpers.

Well, the "easily" part is debatable, nothing seems to be easy with TLS these days. But I get your point.

Does this address your concern?

I would suggest the following change of wording to 2 different sentences

• Be able to perform requests to a specific hostname/ip provided via configuration
• Be able to use infrastructure middleware for inter service communication functionalities including, but not limited to, encryption and circuit-breaking. Alternatively, the service MUST implement those functionalities internally.

I am ambivalent on the MUST, maybe we should relax it to SHOULD.

In T208524#4987398, @jijiki wrote:

Production deployment
<snip>
Have backups if the service stores any data

May be we could add "and a restoration/emergency plan", in terms that sometimes it might not be straightforward of what to do with a backup or what is included in a backup.

ack, done.

In T208524#4987487, @akosiaris wrote:

I would suggest the following change of wording to 2 different sentences

• Be able to perform requests to a specific hostname/ip provided via configuration
• Be able to use infrastructure middleware for inter service communication functionalities including, but not limited to, encryption and circuit-breaking. Alternatively, the service MUST implement those functionalities internally.

I am ambivalent on the MUST, maybe we should relax it to SHOULD.

I like this a lot, I simplified the requirements (and yes I agree on "SHOULD", there are always special cases to consider).

Last Call extended by one week. Now ending at: March 13 11pm PST (March 14 7:00 UTC, 8:00 CET)

Is the canonical location of the text here on task or the wiki?

@greg here in phab

TechCom has approved this

Is this now documented somewhere on mediawiki.org? I don't see it linked from https://www.mediawiki.org/wiki/Development_policy.

@Joe: Do you (or anyone else) know where this is documented?

@Aklapper the RfC has been edited to reflect what's on phabricator at https://www.mediawiki.org/wiki/Requests_for_comment/Standards_for_external_services

I didn't add it to the development policy page though. @daniel I'll create a separate page and link it in the development policy page.

In T208524#5372064, @Joe wrote:

I didn't add it to the development policy page though. @daniel I'll create a separate page and link it in the development policy page.

Sounds good!