Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starter site integration #226

Merged
merged 22 commits into from
Sep 22, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
10fb73a
Allow the installation separate installation of isladnora/openseadrag…
adam-vessey Sep 8, 2022
d59f15e
Theoretically most of it?
adam-vessey Sep 8, 2022
a97bb9a
More granular control over some drupal tasks.
adam-vessey Sep 9, 2022
c420c2b
Add the base starter_dev thing.
adam-vessey Sep 9, 2022
7b8ce49
Allow matomo and fits to not be installed.
adam-vessey Sep 9, 2022
d16d818
Fix new "when" specs, and deconflate Matomo and Solr configs.
adam-vessey Sep 9, 2022
500e8d1
Adjust/expand documentation of the "profiles".
adam-vessey Sep 9, 2022
8f978c2
Fix use of `drupal_enable_modules` variable in starter profiles.
adam-vessey Sep 9, 2022
2137655
Comment out the things being kept around for parallel structures.
adam-vessey Sep 9, 2022
a18bde0
Point at the merged starter site.
adam-vessey Sep 12, 2022
52f962a
Group variables based to some degree on function.
adam-vessey Sep 13, 2022
dd55067
Update the target string accordingly.
adam-vessey Sep 13, 2022
d27c07c
Fix composer invocation to _not_ run as root.
adam-vessey Sep 14, 2022
785491b
Merge remote-tracking branch 'origin/dev' into feature/starter-site
adam-vessey Sep 15, 2022
11cdd49
Back to doing the Matomo and FITS things.
adam-vessey Sep 15, 2022
80e4be7
Since we now do Matomo and FITS, can drop the notes indicating we did…
adam-vessey Sep 15, 2022
61bb5ac
Fix another instance of composer running as root.
adam-vessey Sep 15, 2022
3eba24e
Merge remote-tracking branch 'origin/dev' into feature/starter-site
adam-vessey Sep 16, 2022
632f2af
Add in the other migration.
adam-vessey Sep 20, 2022
54ee90d
Drop `standard` "profile"
adam-vessey Sep 20, 2022
b3f1bca
Point back upstream for the referenced repo.
adam-vessey Sep 20, 2022
2b26252
Include the dev things in the dev thing.
adam-vessey Sep 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,13 @@ By default the Vagrantfile builds Islandora on a `ubuntu/focal64` (20.04 LTS) ba

### Install Profile

The Unix shell variable 'ISLANDORA_INSTALL_PROFILE' can be one of:
The Unix shell variable `ISLANDORA_INSTALL_PROFILE` can be one of:

standard - Installs a drupal/recommended-project base install and enables the Islandora and Islandora Defaults modules without any special configuration.
* `demo`: Installs the demo based on the [install profile](https://github.com/Islandora-Devops/islandora_install_profile_demo) developed by Born Digital. This has a custom theme and more out-of-the-box customizations.
* `starter`: Installs using [the `islandora/islandora-starter-site` project](https://github.com/Islandora/islandora-starter-site/) as a template, intended for spinning up sites for general usage.
* `starter_dev`: Similar to `starter`, installs based on [the `islandora/islandora-starter-site` project](https://github.com/Islandora/islandora-starter-site/); however, performs a clone of the repository with its history, intended specifically for development of the starter site.

demo - Installs the demo based on the [install profile](https://github.com/Islandora-Devops/islandora_install_profile_demo) developed by Born Digital. This has a custom theme and more out-of-the-box customizations.

This corresponds to the 'islandora_profile' Ansible variable.
This corresponds to the `islandora_profile` Ansible variable.

### System Resources

Expand Down
5 changes: 3 additions & 2 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ $virtualBoxDescription = ENV.fetch("ISLANDORA_VAGRANT_VIRTUALBOXDESCRIPTION", "I
# Use 'islandora/8' if you just want to download a ready to run VM that is version 1.1.0 of Islandora
# The 'islandora/8' box is no longer mantained.
$vagrantBox = ENV.fetch("ISLANDORA_DISTRO", "ubuntu/focal64")
# Currently 'standard' and 'demo' are available.
$drupalProfile = ENV.fetch("ISLANDORA_INSTALL_PROFILE", "standard")

# See the "install profile" section of the README for the full gamut available.
$drupalProfile = ENV.fetch("ISLANDORA_INSTALL_PROFILE", "starter")

# vagrant is the main user
$vagrantUser = "vagrant"
Expand Down
6 changes: 6 additions & 0 deletions inventory/vagrant/group_vars/all/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,9 @@ php_packages_state: "latest"
os_environment:
- key: LOG4J_FORMAT_MSG_NO_LOOKUPS
value: true

webserver_install_matomo: true
webserver_install_fits: true

post_install_chown_defaults: true
post_install_migrate_import_fits_tags: false
3 changes: 2 additions & 1 deletion inventory/vagrant/group_vars/webserver/drupal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ drupal_account_name: admin
drupal_trusted_hosts:
- ^localhost$
- "{{ hostvars[groups['webserver'][0]].ansible_host }}"
drupal_trusted_hosts_file: "{{ drupal_core_path }}/sites/default/settings.php"
drupal_site_default_settings: "{{ drupal_core_path }}/sites/default/settings.php"
drupal_trusted_hosts_file: "{{ drupal_site_default_settings }}"
drupal_public_filesystem: "{{ drupal_core_path }}/sites/default/files"
drupal_external_libraries_directory: "{{ drupal_core_path }}/libraries"
fedora_base_url: "http://{{ hostvars[groups['tomcat'][0]].ansible_host }}:8080/fcrepo/rest/"
Expand Down
5 changes: 5 additions & 0 deletions post-install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@
- name: Run migrations
command: "{{ drush_path }} --root {{ drupal_core_path }} -y -l localhost:{{ apache_listen_port }} --userid=1 mim islandora_tags,islandora_defaults_tags"

- name: Run islandora_fits migration
command: "{{ drush_path }} --root {{ drupal_core_path }} -y -l localhost:{{ apache_listen_port }} --userid=1 mim islandora_fits_tags"
when: post_install_migrate_import_fits_tags

- name: Add vagrant user to webserver app user group
user: name={{ vagrant_user }}
group={{ webserver_app_user }}
Expand All @@ -87,6 +91,7 @@
group: "{{ webserver_app_user }}"
mode: 0775
recurse: yes
when: post_install_chown_defaults

- name: Chown controlled access terms default configuration
file:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

openseadragon_drush_path: /usr/local/bin/drush

openseadragon_composer_require: true
openseadragon_composer_item: "islandora/openseadragon:1.0.0"
openseadragon_composer_root: "/var/www/html/drupal"
openseadragon_sites:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
arguments: "{{ openseadragon_composer_item }}"
working_dir: "{{ openseadragon_composer_root }}"
become: no
when: openseadragon_composer_require

- name: Check if Seadragon Library Exists
stat:
Expand Down
9 changes: 9 additions & 0 deletions roles/internal/webserver-app/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,12 @@ solr_user: solr
solr_instance_conf_path: /var/solr/data/{{ solr_cores[0] }}/conf

webserver_document_root: /var/www/html

webserver_app_do_original_trusted_host_and_more: true
webserver_app_do_trusted_host: false
webserver_app_do_fedora_scheme_config: false
webserver_app_do_content_sync_config: false
webserver_app_uninstall_search_module: true
webserver_app_feature_import_base: true
webserver_app_feature_import_search: false
webserver_app_configure_matomo: true
62 changes: 54 additions & 8 deletions roles/internal/webserver-app/tasks/drupal.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---

- name: Trusted Host Settings
- name: Trusted Host Settings and more
adam-vessey marked this conversation as resolved.
Show resolved Hide resolved
blockinfile:
state: present
block: |
Expand All @@ -21,11 +21,53 @@
$content_directories['sync'] = $app_root.'/../content/sync';
path: "{{ drupal_trusted_hosts_file }}"
marker: // {mark} ANSIBLE MANAGED BLOCK
when: webserver_app_do_original_trusted_host_and_more

- name: Trusted Host Settings
blockinfile:
state: present
block: |
$settings['trusted_host_patterns'] = array(
{% for host in drupal_trusted_hosts %}
'{{ host }}',
{% endfor %}
);
path: "{{ drupal_trusted_hosts_file }}"
marker: // {mark} ANSIBLE MANAGED BLOCK - trusted hosts
when: webserver_app_do_trusted_host

- name: Flysystem "fedora" scheme configuration
blockinfile:
state: present
block: |
$settings['flysystem'] = [
'fedora' => [
'driver' => 'fedora',
'config' => [
'root' => '{{ fedora_base_url }}',
],
],
];
path: "{{ drupal_site_default_settings }}"
marker: // {mark} ANSIBLE MANAGED BLOCK - fedora flysystem scheme
when: webserver_app_do_fedora_scheme_config

- name: Content Sync Directory configuration
blockinfile:
state: present
block: |
global $content_directories;
$content_directories['sync'] = $app_root.'/../content/sync';
path: "{{ drupal_site_default_settings }}"
marker: // {mark} ANSIBLE MANAGED BLOCK - content sync dir
when: webserver_app_do_content_sync_config
Comment on lines +24 to +63
Copy link
Contributor Author

@adam-vessey adam-vessey Sep 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the best approach was here, to allow for more granular control over what was being done in what was originally called the "Trusted Host Settings" bit, which was doing more than just setting the "trusted hosts" settings... elected to keep the original, but additionally have all the bits it contained be available in separate chunks, as we don't really want the content/sync thing being done in the starter "profile".



- name: Uninstall core search module
command: "{{ drush_path }} --root {{ drupal_core_path }} -y pm-uninstall search"
register: uninstall_search
changed_when: "'successfully uninstalled' in uninstall_search.stdout"
when: webserver_app_uninstall_search_module

- name: Set permissions on the public files directory
file:
Expand All @@ -48,24 +90,28 @@
path: "{{ drupal_deploy_dir }}/content/sync"
owner: "{{ webserver_app_user }}"
group: "{{ webserver_app_user }}"
when: webserver_app_do_content_sync_config or webserver_app_do_original_trusted_host_and_more

- name: Import features
command: "{{ drush_path }} --root {{ drupal_core_path }} -y fim islandora_core_feature,islandora_defaults"

when: webserver_app_feature_import_base

- name: Import search feature
command: "{{ drush_path }} --root {{ drupal_core_path }} -y fim islandora_search"
when: islandora_profile == "standard"
when: webserver_app_feature_import_search

# Set Matomo Settings
- name: Set Matomo site id.
command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set matomo.settings site_id 1"
register: set_search_api_config
changed_when: "'Do you want to update site_id' in set_search_api_config.stdout"
register: set_matomo_site_id_config
changed_when: "'Do you want to update site_id' in set_matomo_site_id_config.stdout"
when: webserver_app_configure_matomo

- name: Set Matmo sever url.
- name: Set Matomo server url.
command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set matomo.settings url_http http://localhost:8000/matomo/"
register: set_search_api_config
changed_when: "'Do you want to update site_id' in set_search_api_config.stdout"
register: set_matomo_server_url_config
changed_when: "'Do you want to update url_http' in set_matomo_server_url_config.stdout"
when: webserver_app_configure_matomo

# pdf.js library
- name: ensure pdf.js directory exists
Expand Down
4 changes: 2 additions & 2 deletions roles/internal/webserver-app/tasks/solr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
- name: Set default solr server host from hostvars
command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set search_api.server.default_solr_server backend_config.connector_config.host {{ hostvars[groups['solr'][0]].ansible_host }}"
register: set_search_api_config_host
changed_when: "'Do you want to update' in set_search_api_config.stdout"
changed_when: "'Do you want to update' in set_search_api_config_host.stdout"

- name: Set default solr server to point to first core
command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set search_api.server.default_solr_server backend_config.connector_config.core {{ solr_cores[0] }}"
register: set_search_api_config_core
changed_when: "'Do you want to update' in set_search_api_config.stdout"
changed_when: "'Do you want to update' in set_search_api_config_core.stdout"
Comment on lines -9 to +14
Copy link
Contributor Author

@adam-vessey adam-vessey Sep 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking back at https://github.com/Islandora-Devops/islandora-playbook/pull/126/files#diff-5bddb401e18178b7584b51cd9831eaa348385079b2224a85e3db8fa2412fc547L5-R15 ... I believe this is the intent, as otherwise, it seems that the set_search_api_config bit would've been being registered by the "Matomo" setup, in the webserver-app/tasks/drupal.yml bit?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seems like a typo in the Matomo stuff.


- name: Create SOLR config.zip
command:
Expand Down
3 changes: 2 additions & 1 deletion tomcat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,5 @@
- Islandora-Devops.grok
- Islandora-Devops.cantaloupe
- Islandora-Devops.activemq
- Islandora-Devops.fits
- name: Islandora-Devops.fits
when: webserver_install_fits
47 changes: 0 additions & 47 deletions vars/standard.yml

This file was deleted.

31 changes: 31 additions & 0 deletions vars/starter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
drupal_build_composer_project: true

# None necessary, should all be described in the given project.
#drupal_composer_dependencies: []

# Presently pointing at the main branch.
drupal_composer_project_package: 'islandora/islandora-starter-site:dev-main'
Comment on lines +7 to +8
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should eventually be pointed at whatever reference structure we move to use for releases... or maybe left without a version reference to pull whatever "latest" version?


# XXX: Strictly, irrelevant due to using the `--existing-config` flag.
drupal_install_profile: minimal

# None necessary, due to using an existing configuration.
#drupal_enable_modules: []

drupal_site_install_extra_args:
- '--existing-config'

# The "starter" is expected to have all of its composer requirements and config
# included, including spec'ing what modules are enabled in a site.
openseadragon_composer_require: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these - setting up Matomo and FITS - seem like they're false only because theyre out of scope at the moment but we probably eventually want them. Can you denote/comment/group them in some way to distinguish from the others which we do/don't need due to the architecture of the Starter Site?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a shot in 52f962a ... thoughts?

Copy link
Contributor Author

@adam-vessey adam-vessey Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I should mention: Anything that does need to do any configuration probably maybe shouldn't do it by config:set'ing anything, but instead to use other mechanisms such as config_override to avoid potentially dirtying configs... to avoid specific environmental concerns creeping into things; for example: If the matomo.settings:url_http value was made to dynamically point at the hostname instead of just specifying localhost:8000, then yeah, it wouldn't really belong in the config that gets used between environments where that remote would(/could) vary.

... This... does kind of get into a weird area with how the JWT stuff is presently configured, in that because things were based on the "standard" playbook install, that it just so happens to match the paths; however, if other paths are ever specified for webserver_app_jwt_key_path, then it could end up diverging...

webserver_app_uninstall_search_module: false
webserver_app_feature_import_base: false

# We want more granular control over what is added to the "settings.php".
webserver_app_do_original_trusted_host_and_more: false
webserver_app_do_trusted_host: true
webserver_app_do_fedora_scheme_config: true

post_install_migrate_import_fits_tags: true
post_install_chown_defaults: false
33 changes: 33 additions & 0 deletions vars/starter_dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
drupal_build_composer_project: false
drupal_deploy: true
drupal_deploy_repo: https://github.com/Islandora/islandora-starter-site.git
drupal_deploy_version: main
drupal_deploy_dir: "{{ drupal_composer_install_dir }}"
drupal_deploy_composer_install: true

# XXX: Not usable with `drupal_deploy`.
#drupal_composer_dependencies: []

# XXX: Strictly, irrelevant due to using the `--existing-config` flag.
drupal_install_profile: minimal

# None necessary, due to using an existing configuration.
#drupal_enable_modules: []

drupal_site_install_extra_args:
- '--existing-config'

# The "starter" is expected to have all of its composer requirements and config
# included, including spec'ing what modules are enabled in a site.
openseadragon_composer_require: false
webserver_app_uninstall_search_module: false
webserver_app_feature_import_base: false

# We want more granular control over what is added to the "settings.php".
webserver_app_do_original_trusted_host_and_more: false
webserver_app_do_trusted_host: true
webserver_app_do_fedora_scheme_config: true

post_install_migrate_import_fits_tags: true
post_install_chown_defaults: false
3 changes: 2 additions & 1 deletion webserver.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,5 @@
- perms-fix
- Islandora-Devops.drupal-openseadragon
- webserver-app
- Islandora-Devops.matomo
- name: Islandora-Devops.matomo
when: webserver_install_matomo