-
-
Notifications
You must be signed in to change notification settings - Fork 409
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch away from Webpacker to Shakapacker #10389
Conversation
c7bdc4f
to
affda03
Compare
c0f7318
to
e04d936
Compare
The issue with the test assets not compiling automatically can be solved by adding this to the default RSpec configuration: config.before(:all, type: :system) do
Dir.chdir(Rails.root) { Webpacker.compile }
end (note: I understand this will only recompile if the assets have changed) |
OK, I think I've given it enough time, so here are the remaining issues from the previous comments I'd still like to get resolved before merging this:
And remaining issue after this which can be handled in a separate PR, find out a better way to handle the Tailwind configuration generation to avoid overriding the runner (see thread). |
@ahukkanen , all the review recommendations have been applied. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested both development app generation and test app generation and both work just fine now. The ./bin/dev
script works on develop and during tests the assets were correctly compiled when they weren't available.
I have noticed few more issues after the latest changes, could you please have a look?
Also, could you revert the branch name changes that you have marked yourself. It should be good to go after these.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work @alecslupu ! Works really nicely now.
This gives us good time to think where to go from here to improve the assets generation.
* patch the generator to use custom branch * Remove Webpacker, Install Shakapacker * Patch shakapacker decidim-core * Update the rake task and base Js * Update shakapacker npm package * Update npm api * Patch the workflows * Fixing decidim-admin test suite * Fixing decidim-core specs * Fix decidim-dev specs * Fixing decidim-core * Fixing decidim-assemblies specs * Fixing decidim-verifications specs * Fixing decidim-consultations specs * Fix more decidim-assemblies * Fixing decidim-consultations * Fix decidim-conferences * Partially fixing decidim-initiatives * Partially fixing participatory-processes * Fixing decdim-generators * Running linters * Fixing more decidim-core specs * Partially fix decidim-proposals * Fix decidim-accountability specs * Fix decidim-proposal cell * Fix decidim-budgets specs * Fix decidim-debates * Fix decidim-templates * Partially fix decidim-surveys * Fix decidim-surveys * Partially fix decidim-meetings * Fix decidim-templates * fix geocoding fields * Fix more decidim-initiatives * Fix more decidim-meetings specs * Fix more decidim-proposals * Fix more decidim-meetings * Fix decidim-elections, decidim-votings (decidim#10422) * Fix decidim-elections, decidim-votings * fix more decidim-elections * Fixing more decidim-elections * more fixes in decidim-elections * Add release notes * Lint RELEASE_NOTES.md * Set custom webpacker.yml location * Patch lighthouse workflow * Remove unecessary content_fors * Patch the webpacker config * Make use of append_stylesheet_pack_tag * Fixing the map elements * patch the release notes * Lint erb files * Apply MR recommendations * Patch rails cells to use the rails view context as a reference * Replace javascript_snippets and stylesheet_snippets by append_assets * Update to shakapacker 6.6.0 * Remove shakapacker override in generator * Remove workflow patch * Linters * Fix generator * Bugfix * Revert commonmarker downgrade * Apply review recomandations regarding procfile * Apply review recommendations * Lock both JS and Ruby shakapacker versions at 6.6.x to avoid breaking our pipelines * Run the decidim:procfile:install rake task during application generation * Chmod the bin/dev script correctly * Change the default for WEBPACKER_RUNTIME_COMPILE to false * Compile the assets automatically during tests if they haven't been compiled before * Remove unnecessary lines from the map specs after the refactoring * Lock both JS and Ruby shakapacker versions at 6.6.x to avoid breaking * Preparing to merge to develop * Adding development notes, as suggested * Last batch of changes
* Integrate the editor with the redesigned layout * Add a general upload error message on non HTTP 200 responses * Remove done item from the comment * Fix async issue with the image upload handling The image upload response handling was not always returning a promise which caused inconsistent behavior across browsers. * Normalize i18n * Remove the quill list styling hacks * Remove legacy quill styling classes from CSS * Modify the iframe disabler tests to use new video embed markup * Remove vendored legacy Quill extensions * Remove the TipTap YouTube extension as it is not used * Implement image resizer * Add dialog extension and refactor image resizing - In order to control the editor styling while the modal dialogs are open, add a new dialog extension that stores the dialog state for the editor itself - Refactor the image extension to move the custom node view for image resizing to its own file * Add activity outline for editor images and video embeds * Move the other extensions to a similar folder structure In order to maintain consistency for the structuring of the code, move other extensions to a folder stucture after the image extension required its own folder. * Add support for user mentions and hashtags * Fix checking if the src has changed The `img` element's src can also get the hostname prepended to it when the previous check would not work. Updated to check if the actual node attribute has changed. * Remove console logs * Fix the editor system specs for image content after image resizer As the generated editor HTML changed, the expectation also needs to be changed. * Fix editor hashtags suggestion list label rendering * Fix mentions sorting strings (exclude the mention character) * Conditionally add hashtag and mention extensions to the editor In case the editor container contains the classes to indicate these features are needed, only add those extensions in that case. * Do not attach tribute to the editor The editor implements its own hashtags and mentions functionality which is why we do not need to attach tribute to the editors. * Remove tribute styling from the editor as it is not used * Remove Quill global from the default eslint config * Match design app packages * Integrate character counter with the editor * Fix the enabled/disabled editor toggle in conferences and meetings In case the registrations are disabled, disable also the editors in those views. Previously they were only enabled. * Remove the last TODO item * Rename editor modal to dialog for consistency We are using the dialog concept in the redesign to control the dialogs, so we want to use consistent naming also in the editor. * Fix the form builder upload dialog selector attribute * Add `allowfullscreen="true"` to the iframe embeds * Add content migrator to migrate WYSIWYG content This handles the changes in the WYSIWYG content formatting between Quill.js and TipTap by converting the legacy format to the format generated by the new TipTap editor. * Fix sub-list margins We do not want to add the paragraph margin in sub-lists. * Allow content migration to be run on already migrated content * Add methods to migrate the actual data + specs for them * Add the WYSIWYG migration rake task * Fix editor input height issues The height was set on the whole container that also includes the toolbar but it should be actually on the input area instead. * Editor CSS in Tailwind * Refactor decidim_editor to its own JS pack for CSS compatibility In order to utilize the Tailwind CSS based editor CSS, we include it in the JS and pack the editor as its own includable asset. * Update JS override docs reference to the `editor.js` * Add entrypoint for the editor CSS and include it in the head * Remove the editor CSS include from the index.js Because it is already included in the entrypoint. * Integrate the indent extension with lists * Remove the TODO after applying Tailwind to the editor CSS * Add the `redesigned_` prefix to the editor CSS to avoid lint issue * Do not include the legacy (removed) editor.scss Remove the legacy editor.scss import from legacy design and decidim-system. * Fix the editor CSS custom properties/colors + suggestions properties * Fix list integration with the indent extension * Rewrite the indent extension to simplify its code To better utilize the TipTap APIs instead of relying more on ProseMirror, rewrite the indent extension to also consider the Decidim specific functionality better, i.e. managing the indentation through the toolbar. * Add editor related release notes * Fix the content migration registration task's description * Fix button order in the input dialogs under redesigned layout * Also display the suggestions on start and change innerText to textContent * Change `innerText` to `textContent` to support jest tests * Consider the toolbar styles for adding the editor features We have `full` and `basic` toolbars that support different features. Consider this when initiating the editor. * Do not search hashtags if the query is less than 2 characters * Add jest tests for the editor * Fix accessibility violations with the editor and add labelledby * Use the `uniqueId` helper also in the input dialog * Fix empty target on the link element * Add E2E system/capybara tests for the editor * Add ID to the editor elements after the a11y/validation fixes After making the validation fixes, the editor elements were no longer targetable in the specs because of the missing ID. * Fix the targeting of the editor elements in specs * Fix new proposal specs after changes in the editor structure * Fix the missing title elements for the video embed pasting * Add test for hashtag with the Enter key * Improvement on test helpers range selector, basic editor and refactor * Add tests for the indent extension * Add tests for the VideoEmbed extension * Toggle the dialog state correctly for the image upload dialog * Add tests for the link extension * Improve the file related test helpers for the image extension tests * Add tests for the image extension * Add the Enter key test for the mention extension * Remove commented code * Add E2E tests for video embed and image toolbar controls * Add E2E tests for pasting content * Name the describe block correctly in the Image extension test * Add tests for the image resizing * Extend the image extension tests to cover the touch dragging * Add E2E tests for image resizing and fix E2E tests image endpoints * Add E2E tests for the legacy designed editor * Add jest tests for editor with the legacy design * Add width and height info to added images * Fix width setting on first image load to the editor * Fix onload image text issue * Fix a bug in pasting image * Add setTimeout to fix initial value issue in test * Add tests for dimensions text * Add system test for image dimensions info * Fix linting error * Fix js test fails * Remove commented code * Add code comments to describe some of the image node view oddities * Fix the admin manages organization spec after adding image dimensions * Add prose styling to the editor content * Remove the customised CodeBlock extension and its markup After applying the prose styling to the editor content, the custom `code-block` class is no longer necessary on the code block elements. * Fix the image and video embed margins after applying prose styling * Fix image resizer on touch screens and add accessibility labels * Apply the image resizer changes to the admin manages org spec * Fix initiating hashtags and mentions capability The capability classes were checked against the wrong container. * Parse and render hashtags and mentions correctly at the backend Because Tiptap requires a specific format for the hashtags and mentions, we convert them for the editor specifically and translate them back to the GID references at the backend before the content is saved to the database. * Fix a sanitize helper bug when the list elements have line breaks In case the list elements are prepended or appended with line breaks, sanitizing HTML to text does not work correctly. * Dispatch the emoji event on the correct element Previously the element was not properly dispatched on the contenteditable elements. * Add emoji support for the editor * Fix the emoji input contenteditable and editor integration * Fix the color of the disabled emoji button * Fix input emoji positioning with the content editable elements * Fix hashtags on the participant side with WYSIWYG editor * Handle the `rel` attribute on the links through LinkRenderer In order to control when we add the necessary `rel` attributes for the user entered links, we control this through the LinkRenderer at the backend and leave these attributes out of the original link elements. * Fix the `rel` expectations in the Jest test for the editor * Fix the system specs after the `rel` changes to the editor * Mock `@pickmo/popup-picker` for the specs due to incompatibility * Fix syntax issue with the Emoji extension * Fix autolinking links within HTML content In case the link was without the link tag in HTML content, it was not autolinked properly. * Fix rendered link expectations after unifying their attributes * Fix redenred content expectations after changing linkrenderer behavior * Fix link renderer spec after changes in the link href format After the link is generated through the tag helper, some characters are now escaped in the `href` attribute. * Fix heading extension compatibility with hashtags extension * Fix typo * Remove console logs * Move similar spec code to a shared context * Remove commented spec * Support mentions and emojis for the editor through form builder * Add system specs for the editor hashtags and mentions * add-image-button-fix/test * Also test adding files in the redesign + small refactor for the spec * Fix the user mentions spec in the editor E2E tests * Add editor spec for selecting emojis * Fix character counter bug with new paragraph breaks after reaching limit * Add jest tests for the editor Emoji extension * Add jest test for the editor Heading extension * Fix image resizing on Firefox * Do not use the MouseEvent class after adding the device information * Fix the ordered list types/styling * Fix the test name for the Heading extension test * Fix the image resize specs after changes in the resizer * Disable the link target control from participant facing editors * Change the link control and dialog behavior slightly - Take out the "Remove" button from the link dialog to make it consistent with the other dialogs - Make the toolbar link control remove the existing link when an existing link is selected - Allow double clicking an existing link to open the link edit dialog * Simplify * Update the minimum versions of Tiptap * Allow editing and removing link from toolbar - Re-allow editing the link from the toolbar link button - Allow removing the link using the erase styles toolbar button * Implement link bubble * Fix a11y issue with `aria-expanded` attribute added by tippy.js * Fix pasting bold list items with unbold sublist items from google docs * Fix ordered list style for Google Docs * Remove console.log * Fix bug in the input dialog not toggling dialog mode off When the dialog was closed through some other method than clicking one of the dialog actions, it did not toggle the dialog mode off properly. * Rework the link bubble styles, prevent text overflow * Remove unnecessary commented code * Generalize the bubble menu implementation Separate the specific functionality for the link bubble menu and the rest of the bubble menu code in order to make the component useful also for other possible use cases. * Change the legacy dialog tests after changing input dialog JS Trigger the `closed` event on the dialog element in order to make the legacy dialog tests work with the changes. * Move the override methods to the bottom of the class * Remove `tippy.js` dependency in favor of custom implementation Instead of introducing a new rather heavy dependency (for its purpose) for managing the bubble menus, implement the functionality ourselves. * Refactoring: introduce decidim kit extension - Move some of the initialization logic to the DecidimKit extension - Move the upload dialog initialization to the image extension due to the refactoring effort - Fix the jest tests for the image extension * Name the extension correctly * Move image extension options to the correct if clause * Remove unnecessary commented code from the test * Add keyboard shortcuts to change the ordered list type Add keyboard shortcuts (`ALT`+`SHIFT`+`UP` and `ALT`+`SHIFT`+`DOWN`) to navigate through the ordered list types. * Name the test example correctly * Add tests for the added keyboard shortcut for changing ordered list type * Add tests for the clipboard pasted ordered lists Test that some issues discovered during the acceptance testing are properly handled by the editor. This relates to pasting ordered lists with the `type` defined for them which defines the list item styling (e.g. alphabetical lists and roman numeral lists). * Add reference to the TipTap issue regarding character counter * Modify the Google Docs expectation according to actual markup When contnet is copied from Google Docs, the sub-lists are outside of the list elements they are decendant of. Modify the pasted content according to this. * Add reference to the TipTap GitHub issue * Add Jest test for the Google Docs formatted ordered list * Add reference to the TipTap GitHub issue for bold content in lists * Fix styling ordered lists with the `type` attribute * Change the typed ordered list style definitions to Tailwind syntax * Fix pasting lists from Office 365 * Add jest tests for the DecidimKit extension * Add tests for the bold extension customization * Correct the markup example in the comment * Correct the extension imports in the jest tests * Improve the code for processing Office 365 pasted content * Fix pasting lists from desktop Word * Lock sass-embedded to 1.57.1 due to a compile bug * Remove unused code * Use upload templates in the image extension jest test Instead of creating the upload dialog element manually, use the upload templates available in the fixtures. * Do not add `null` values for the list `type` attribute * Only add the `type` attribute to `<ol>` lists * Test the single list item flat list copied from desktop Word As the single items are a special case, test those as well. Also to test that the bullet lists are correctly transformed to `<ul>` elements. * Fix external link indicators triggering within the editor in redesign * Add tests for the link bubble menu * Fix updating some input dialog values using ENTER Submitting all input dialogs did not work using the ENTER key. This makes the behavior consistent over all input dialogs. * Fix flaky editor system specs regarding the bubble menu * Fix rubocop violation in the editor spec * Fix the disabled rubocop rule namespace * Remove unnecessary if clause * Fix form submit when resizing an image on certain forms * Fix jest expectations after adding `type` to image resize controls * Fix the expectation at the admin manages organization spec After adding the `type="button"` to the image resize controls, the expectation at the admin manages organization spec was incorrect. * Enable new spelling checks (#10605) * Switch away from Webpacker to Shakapacker (#10389) * patch the generator to use custom branch * Remove Webpacker, Install Shakapacker * Patch shakapacker decidim-core * Update the rake task and base Js * Update shakapacker npm package * Update npm api * Patch the workflows * Fixing decidim-admin test suite * Fixing decidim-core specs * Fix decidim-dev specs * Fixing decidim-core * Fixing decidim-assemblies specs * Fixing decidim-verifications specs * Fixing decidim-consultations specs * Fix more decidim-assemblies * Fixing decidim-consultations * Fix decidim-conferences * Partially fixing decidim-initiatives * Partially fixing participatory-processes * Fixing decdim-generators * Running linters * Fixing more decidim-core specs * Partially fix decidim-proposals * Fix decidim-accountability specs * Fix decidim-proposal cell * Fix decidim-budgets specs * Fix decidim-debates * Fix decidim-templates * Partially fix decidim-surveys * Fix decidim-surveys * Partially fix decidim-meetings * Fix decidim-templates * fix geocoding fields * Fix more decidim-initiatives * Fix more decidim-meetings specs * Fix more decidim-proposals * Fix more decidim-meetings * Fix decidim-elections, decidim-votings (#10422) * Fix decidim-elections, decidim-votings * fix more decidim-elections * Fixing more decidim-elections * more fixes in decidim-elections * Add release notes * Lint RELEASE_NOTES.md * Set custom webpacker.yml location * Patch lighthouse workflow * Remove unecessary content_fors * Patch the webpacker config * Make use of append_stylesheet_pack_tag * Fixing the map elements * patch the release notes * Lint erb files * Apply MR recommendations * Patch rails cells to use the rails view context as a reference * Replace javascript_snippets and stylesheet_snippets by append_assets * Update to shakapacker 6.6.0 * Remove shakapacker override in generator * Remove workflow patch * Linters * Fix generator * Bugfix * Revert commonmarker downgrade * Apply review recomandations regarding procfile * Apply review recommendations * Lock both JS and Ruby shakapacker versions at 6.6.x to avoid breaking our pipelines * Run the decidim:procfile:install rake task during application generation * Chmod the bin/dev script correctly * Change the default for WEBPACKER_RUNTIME_COMPILE to false * Compile the assets automatically during tests if they haven't been compiled before * Remove unnecessary lines from the map specs after the refactoring * Lock both JS and Ruby shakapacker versions at 6.6.x to avoid breaking * Preparing to merge to develop * Adding development notes, as suggested * Last batch of changes * Further spelling mistake fixes (#10576) * Add further spelling mistake fixes * Fix the label for the admin TOS field * Add missing dots at the end of the sentences * Apply suggestions from code review Co-authored-by: Andrés Pereira de Lucena <[email protected]> * Address crowdin issue --------- Co-authored-by: Andrés Pereira de Lucena <[email protected]> * Standardize the format of the words "those will" (#10619) * Add Social Share Modal missing image to documentation (#10612) * Fix flaky collaborative drafts specs (#10601) * Fix flaky collaborative drafts specs * Make sure the collaboration request was accepted before relogin * Expect that the access request has been successfully sent * Expect the correct page before re-visiting it * Standardize the format of the words "I will" (#10618) * Fix memory leak with user answers serializer (at survey export) (#10418) * Fix memory leak with user answers serializer (at survey export) * Simplify the answer generation * Fix the spec after word standardization (#10624) * Standardize the format of the words "they will" (#10617) * Upgrade Graphql to 2.0.19 and Graphql-Api to 3.0.1 (#10606) * Upgrade Graphql to 2.x.x and Graphql-Api to 3.0.1 * Add Release Notes information * Revent ruby version change * Update jest, babel and related NPM packages * Fix the jest tests * Fix the application helper specs * Fix the editor asset includes with shakapacker * Fix package-lock.json mismatch * Fix package-lock.json mismatch * Fix lint error * Run prettier * Stylelint * Update tiptap packages to the official release version * Disable the heading selector for the `basic` type editor * Fix the jest tests after removing heading from the basic toolbar And add the tests for the content toolbar. * Adjust the system editor specs according to the heading control changes * Update the JS config files during decidim upgrade rake task * Move the front-end test helpers to generalized modules To be able to utilize the front-end helpers in other tests as well. * Raise an error in the wysiwyg migrator if the column is not defined * Add a comment explaining the return statement * Correct the changed column name in the wysiwyg migration * Change the `puts` statements to `log_info` * Add specs for the wysiwyg migration rake tasks --------- Co-authored-by: Sina Eftekhar <[email protected]> Co-authored-by: Joonas <[email protected]> Co-authored-by: Andrés Pereira de Lucena <[email protected]> Co-authored-by: Alexandru Emil Lupu <[email protected]>
🎩 What? Why?
Decidim has migrated from sprockets to Webpacker in #7464, almost 2 years ago. Since then, the rails team has decided to retire the webpacker
Based on the discussions we had in the thread: #8783 we could not find a consensus on what is the best way of approaching the webpacker retirement, yet, since there are no new releases to webpacker, and since there are bugfixes and security improvements happening javascript ecosystem, we need to make upgrades so we can keep a high standard to the application.
I have chosen to migrate to Shakapacker, because we need to perform upgrades to the system, and Shakapacker is the only tool that offers backwards compatibility with 0.26 and 0.27. I did not liked the ideea in the first place, based on the arguments that have been provided in the discussion, yet, now, seems more stable and more promising that the current system.
Upgrade and Testing
In order to perform the update, you will need to make sure that you do not have webpacker in your Gemfile.
If you have it, please remove it, or change it to
Please backup the following files, to make sure that you save any customizations you may have done to webpacker:
After you have performed the backup, make sure you run:
Then run the below command, and replace all the configuration with the one that Decidim is providing by default:
Compiling the JS and CSS at the runtime is a very slow process. Additionally, compiling JS and CSS in the test suite will cause your pipeline to timeout.
To avoid that, you could export the following env variable:
Then use the command to watch and compile the javascript and css upon changes: