Although for my plugin reviews I do not do a comprehensive analysis, this is something I use in my consulting and training and recommend to people to think about these type of issues when considering implementing a 3rd party plugin.
There are many great 3rd-party plugins available for Moodle. However, it is important to assess the suitability and reliability of the plugin before adopting it. The list in the linked document is not exhaustive as there may be more or fewer questions depending on the individual installation and organisation.
Some hosting providers may have already have audited the plugin and publish a list of ones they approve for their hosting platform - Remote-Learner for example. However. as although something may be technically okay, the suitability and cost implications may be something to make you think.
The list is basically something that one could use to assess the risk related to implementing the plugin. Each question could have a few lines to describe what it means, but you should get the overall picture. To give some context I will explain the reasoning behind three of the questions here:
Question: Has it got a Moodle Docs page.
Even though a plugin may have extensive documentation already, if someone is in Moodle and clicks the Moodle Docs link at the bottom of the page in a plugin that’s where they end up. So having a page, or even a stub page which has the links to official documentation is a good thing and helps people getting lost.
Question: Do they (the author)work for an established Moodle development team (Moodle partner, the OU, or HQ for example)?
This is an assessment of business risk. If someone is a freelancer, the ability for them to provide ongoing support for a module is sometimes (not always) less than where it was built-in an established development team where it may be in use by many clients or installations and thus maintained for business reasons. This is not always the case, but it is just one of the many things that one coud consider.
Question: Does it impact performance of Moodle in normal or high traffic usage?
How complex is the solution? Is it using a lot of slow database queries constantly, are they optimised? are the server resources being heavily used by this and reducing the overall performance? Does it encourage more synchronous activity or is it asynchronous? Understanding what the impact of the usage of the plugin, especially if it is heavily adopted by courses is important.Download Some Criteria for reviewing plugins (pdf) - 409.64 kB
Please feel free to add any questions or sections that you think should be added to the list