Rc1 - #1765

Closed

wortmanb wants to merge 286 commits into

elastic:masterfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 15, 2025

Refactored deepfreeze
Bret Wortman
committed
532adb9
Updates & fixes to click plumbing
Bret Wortman
committed
5a5e366

Commits on Jan 16, 2025

Added save_settings
Bret Wortman
committed
6396899

Commits on Jan 17, 2025

Still trying to get the plumbing right
Bret Wortman
committed
be61cc3

Commits on Jan 24, 2025

Fix extra paren
Bret Wortman
committed
7fb4934
Fix action dispatching
Bret Wortman
committed
891ab82

Commits on Jan 26, 2025

Setup works
Bret Wortman
committed
dab6f38

Commits on Jan 27, 2025

Update README.rst
wortmanb
authored
22ac9a3
Today's big update
Bret Wortman
committed
5612c51
Merge branch 'master' of github.com:wortmanb/curator
Bret Wortman
committed
e987730
Purging old repos
Bret Wortman
committed
d33c035

Commits on Jan 28, 2025

Fixed selection of the latest repo
Bret Wortman
committed
9218ef0
Code cleanup
Bret Wortman
committed
a3961da
Trying to make Flake happy
Bret Wortman
committed
a7e68fe
Doco & formatting fixes
Bret Wortman
committed
ebfaf19
Added two more dataclasses
Bret Wortman
committed
1732905

Commits on Jan 29, 2025

First unit tests
Bret Wortman
committed
fb350c3
More tests
Bret Wortman
committed
cfaa6d3
Beginning to notate types
Bret Wortman
committed
08ce133
Test suite for DF Setup
Bret Wortman
committed
f803c75
Muzzling the linter
Bret Wortman
committed
c5c6ca0
test suite for Rotate
Bret Wortman
committed
b9cb94e

Commits on Jan 30, 2025

Early work on thawing
Bret Wortman
committed
14b14c1
Adding ruff.toml
Bret Wortman
committed
9bb6317
Making Ruff happy
Bret Wortman
committed
02bd510
Mostly added type hints
Bret Wortman
committed
3da6be7

Commits on Jan 31, 2025

Added s3client.py
Bret Wortman
committed
63a1cdb
Refactored to use new S3Client class
Bret Wortman
committed
df861e9

Commits on Feb 1, 2025

Unit test for S3Client
Bret Wortman
committed
a9c8727

Commits on Feb 3, 2025

Removing unit test for obsolete method
Bret Wortman
committed
41bbded
Fixed some unit tests
Bret Wortman
committed
5cca7c8
Cleaned up imports
Bret Wortman
committed
4800010
Updates to ThawedRepo
Bret Wortman
committed
21a3779

Commits on Feb 4, 2025

Test code to determine earliest/latest @timestamp in repo
Bret Wortman
committed
1ca7aab

Commits on Feb 5, 2025

Added get_timestamp_range
Bret Wortman
committed
4acdb9d
Expanded use of new get_timestamp_range
Bret Wortman
committed
73abe95
Re-templating deepfreeze integration testing
Bret Wortman
committed
9f14b33
rough outline of setup integration test
Bret Wortman
committed
e2ad76d
Removing repo_time_tester.py
Bret Wortman
committed
bc74a84
Templated new methods in s3client.py
Bret Wortman
committed
87db906
Fix unused import in integration test
Bret Wortman
committed
f1e4e78
Implement index thawing
Bret Wortman
committed
45a6004
Doco fixes
Bret Wortman
committed
7818075

Commits on Feb 6, 2025

Updates to latest methods
Bret Wortman
committed
eca85a4
New tests
Bret Wortman
committed
7273d53

Commits on Oct 10, 2025

Merge branch 'master' of github.com:elastic/curator
Bret Wortman
committed
cd1ea0a
Status action
Bret Wortman
committed
7e5ebb1
Refactored
Bret Wortman
committed
613ec4f
Add rich to the dependency list
Bret Wortman
committed
186c348
Switching to rich for interactive output
Bret Wortman
committed
85654ce
Adding to .gitignore
Bret Wortman
committed
7477988
Another script of mine that doesn't need sharing
Bret Wortman
committed
2abeddd
Type hints
Bret Wortman
committed
86f0629
Repository record-keeping
Bret Wortman
committed
ec54d59
Starting to flesh out thawing
Bret Wortman
committed
e508062
Adding cluster name
Bret Wortman
committed
aeb48cd
Another script of mine that doesn't need sharing
Bret Wortman
committed
14eb009
Remove stale method
Bret Wortman
committed
4f42301
Removing ancient version of thaw.py
Bret Wortman
committed
267a15e
Merge branch 'master' of github.com:wortmanb/curator
Bret Wortman
committed
a57d1af
FIxing options, testing setup
Bret Wortman
committed
a5ce331
Adding push to glacier
Bret Wortman
committed
471c509
Refafctored a couple utility methods
Bret Wortman
committed
6c826c2
Add check_restore_status method
Bret Wortman
committed
fc76805
This is worth keeping
Bret Wortman
committed
b6e0b21
Adding Remount
Bret Wortman
committed
f3725fb
Refactored object name
Bret Wortman
committed
dd446a5
Fleshing out new actions
Bret Wortman
committed
05badf0
Updated unit tests for utility functions
Bret Wortman
committed
81034e0
Tests now work
Bret Wortman
committed
d898eec
Switching back to dictionary-like interface
Bret Wortman
committed
bf5c5e1
Added list_objects method
Bret Wortman
committed
e6f2f1d
Loads of updates
Bret Wortman
committed
ee9682b
Update Repository to construct from name or dictionary
Bret Wortman
committed
249650d
Add methods for use in testing, mostly
Bret Wortman
committed
f5beefd
Added method to update mounted repo date ranges
Bret Wortman
committed
325612d
Changes to methods with better signatures
Bret Wortman
committed
8bf440b
Docstring updates
Bret Wortman
committed
5d211f4
Merging change to modularize deepfreee
Bret Wortman
committed
a439d57
Removing unneeded import
Bret Wortman
committed
664fce2
Minor tweaks
Bret Wortman
committed
7548c69
Adding exceptions specific to Deepfreeze
Bret Wortman
committed
8c65d69
Expand ensure_settings_index definition
Bret Wortman
committed
dc6bd02
Flesh out integration tests for rotate
Bret Wortman
committed
35b2251
Adding more exceptions
Bret Wortman
committed
c017f74
Adding readme to track work still to be done
Bret Wortman
committed
adff5cd
Move the geneeration of new Repository objects outside the object
Bret Wortman
committed
6b095ad
Fixed issues with rotate testing
Bret Wortman
committed
a566227
Increase interval to remove race conditions
Bret Wortman
committed
b4a1fa2
Updates from expanded integration testing
Bret Wortman
committed
0f92edf
New test cases to better cover repo/bucket issues
Bret Wortman
committed
b6dbaa2
Removing integration test for status
Bret Wortman
committed
f7fad72
Templates which can give success right now
Bret Wortman
committed
fe6b8e9
Spelling matters
Bret Wortman
committed
0679423
Add snapshot count to repository status
Bret Wortman
committed
1234a27
Update repos after creating the latest, not before
Bret Wortman
committed
85e4871
Various testing updates
Bret Wortman
committed
006e5eb
Updates to Repository utilities
Bret Wortman
committed
eb84d9c
Type hints
Bret Wortman
committed
fae0d47
Merge branch 'deepfreeze' of github.com:wortmanb/curator into deepfreeze
Bret Wortman
committed
565271b
Removing unused imports & assignments
Bret Wortman
committed
4fc0bff
Restored setup functionality with boundaries
Bret Wortman
committed
3516d6a
Fixing isoformat error
Bret Wortman
committed
e5c3468
Fix repo searching
Bret Wortman
committed
f902b9a
Fixing some issues with repo rotation
Bret Wortman
committed
4dd1165
Added single-letter options to al actions
Bret Wortman
committed
4c9fddc
Adding testing setup scripts
Bret Wortman
committed
1899b27
Documentation update
Bret Wortman
committed
217cb77
Fixes for date comparisons
Bret Wortman
committed
3e43c15
Fixed AWS code, repo removal
Bret Wortman
committed
203f815
Fixing date updating
Bret Wortman
committed
35f40f9
Do this query using name, not id
Bret Wortman
committed
5bf7b99
Remove unnecessary import
Bret Wortman
committed
cd15ac3
Fix missing base_path and bucket
Bret Wortman
committed
fc3701c
Fixed repo with null start/end
Bret Wortman
committed
dc4b1bb
Clean up
Bret Wortman
committed
42accb0
Facilitate screening repos by mounted status
Bret Wortman
committed
5920a5f
Removing all but the options I've been working on most. We'll add capability in future releases. Focus on MVP for now. Also show defaults in help text.

wortmanb
authored and
Bret Wortman
committed
6b1024a
Bugfix; change approach to unmounting failures
Bret Wortman
committed
cecef8c
Merge branch 'dev' of github.com:wortmanb/curator into dev
Bret Wortman
committed
252b61b
Update to ILM policy for faster testing
Bret Wortman
committed
25daf1d
Bugfixes
Bret Wortman
committed
1b2af2c
Updates to reflect renaming of this function
Bret Wortman
committed
4856fb4
Removing actions
Bret Wortman
committed
c52d10e
More removals
Bret Wortman
committed
d1085d5
Cleaning up
Bret Wortman
committed
89e70c5
More aggressive ILM policy
Bret Wortman
committed
f50793a
Change these warnings to infos
Bret Wortman
committed
94e9950
Removing unneeded import
Bret Wortman
committed
38cfe68
More aggressive ILM policy for testing & demo
Bret Wortman
committed
4b21fc0
Don't limit ourselves needlessly
Bret Wortman
committed
52c454d
Removing FIXMEs and TODOs that have been handled
Bret Wortman
committed
4dd7b0a
More code removal
Bret Wortman
committed
1ca6bc9
Implement S3Client as an ABC
Bret Wortman
committed
91d652e
WIP on Glacier
Bret Wortman
committed
862765b
Updating to fix problem with bucket/path contents not aging to Glacier.
Bret Wortman
committed
e750c05
Removing an unnecessary import
Bret Wortman
committed
6bc926f
Updated to work with nginx reverse proxy
Bret Wortman
committed
6fafcff
Fixing misspellings of "Deepfreeze"
Bret Wortman
committed
99f4e98
Removing files used for local testing
Bret Wortman
committed
ebb8cdf
Fixing missing commas
Bret Wortman
committed
9979909
Unit testing
Bret Wortman
committed
aff2afe

Commits on Oct 12, 2025

Remove warnings, re-add actions to those who don't use filters
Bret Wortman
committed
04e85b7

Commits on Oct 13, 2025

Fix repo date range calculation & storage
Bret Wortman
committed
19fa35a
First pass at Thaw action
Bret Wortman
committed
6cc4363
Adding thaw id management and listing
Bret Wortman
committed
b476802
Post-rebase updates
Bret Wortman
committed
627df7c
Date logic updates
Bret Wortman
committed
296125e
Add cleanup action
Bret Wortman
committed
125a1ea
Add --limit to status (show only _n_ repositories)
Bret Wortman
committed
a541e84
Fixup cleanup CLI and tweak help messages
Bret Wortman
committed
8d7f226
Cleanup indices as well as repositories
Bret Wortman
committed
b006304
Added refreeze
Bret Wortman
committed
e57f482
Added refreeze by thaw_id, and get confirmation before destructive actions.
Bret Wortman
committed
4687957
How to manage the various git branches
Bret Wortman
committed
00e1186
Merge branch 'RC1' of github.com:wortmanb/curator into RC1
Bret Wortman
committed
e3671a8
Improve error messages during deepfreeze setup
Bret Wortman
committed
b617705
Improve error messages during deepfreeze setup
Bret Wortman
committed
8eb289c

Commits on Oct 14, 2025

[BF] Handle multiple policies during rotation
Bret Wortman
committed
86c46b0
Update unit tests
Bret Wortman
committed
01e70d4

Commits on Oct 15, 2025

Merge branch 'RC2' of github.com:wortmanb/curator into RC2
Bret Wortman
committed
0bb97ad
Fixed detection of initial repo(s)
Bret Wortman
committed
bf767ae
Merge branch 'RC2' of github.com:wortmanb/curator into RC2
Bret Wortman
committed
d9942fa
Type hints
Bret Wortman
committed
d79ea7d
Escape bucket paths and other fixes
Bret Wortman
committed
02bda0c
First pass at Thaw action
Bret Wortman
committed
6f61d10
Adding thaw id management and listing
Bret Wortman
committed
788b0fc
Post-rebase updates
Bret Wortman
committed
7a44dcb
Date logic updates
Bret Wortman
committed
52851ac
Add cleanup action
Bret Wortman
committed
946dc15
Add --limit to status (show only _n_ repositories)
Bret Wortman
committed
9a2fdcc
Fixup cleanup CLI and tweak help messages
Bret Wortman
committed
462fad4
Cleanup indices as well as repositories
Bret Wortman
committed
947a687
Added refreeze
Bret Wortman
committed
a6c6fab
Improve error messages during deepfreeze setup
Bret Wortman
committed
964dad3
Added refreeze by thaw_id, and get confirmation before destructive actions.
Bret Wortman
committed
a1fbc38
[BF] Handle multiple policies during rotation
Bret Wortman
committed
658cf14
Improve error messages during deepfreeze setup
Bret Wortman
committed
a5df42e
Update unit tests
Bret Wortman
committed
26ef48b
Merge branch 'RC2' of github.com:wortmanb/curator into RC2
Bret Wortman
committed
83ba522
Fixed detection of initial repo(s)
Bret Wortman
committed
9670c9b
Merge branch 'RC2' of github.com:wortmanb/curator into RC2
Bret Wortman
committed
7792dd2
Type hints
Bret Wortman
committed
af5e282
Escape bucket paths and other fixes
Bret Wortman
committed
a0f7b8a
Merge branch 'RC2' of github.com:wortmanb/curator into RC2
Bret Wortman
committed
d4711c4
Added doc to help explain the process
Bret Wortman
committed
9427287
Fail quickly if no ILM policies reference the repo
Bret Wortman
committed
b5e42cc

Commits on Oct 16, 2025

Improve messaging during thaw operations
Bret Wortman
committed
51e8cc7
Fix display of thawing and thawed repos in status output
Bret Wortman
committed
b5d3932
Add section selections & porcelain output
Bret Wortman
committed
50ad269
Show full Request ID
Bret Wortman
committed
bee6f82

Commits on Oct 17, 2025

Print thaw ID after async creation
Bret Wortman
committed
5cef7ca
Cleanup old thaw IDs
Bret Wortman
committed
43aece5
Fix repo thaw check
Bret Wortman
committed
bfcbe76
Restore snapshots and indices, too
Bret Wortman
committed
6a1f5cb
Speed improvements
Bret Wortman
committed
fdfe60b
Fix thawed repo detection in status
Bret Wortman
committed
5f8f805
Move thawed to their own section in status
Bret Wortman
committed
e207590
Protect thawed repos during rotation
Bret Wortman
committed
04585af
Add date range to thaw --list
Bret Wortman
committed
39e70b9
thaw --check-status without an ID checks all
Bret Wortman
committed
57e3b05
Critical Bugfix
Bret Wortman
committed
88f1cae

Commits on Oct 18, 2025

Fixing logic for updating date ranges when thawed indices are mounted
Bret Wortman
committed
c3a11b9
Fixup script for bad repo dates
Bret Wortman
committed
3d7f7bd
New state machine for thawed repos
Bret Wortman
committed
b204b5f
Implement new state machine
Bret Wortman
committed
46c2a59
Fix repo state issue
Bret Wortman
committed
824eae8
Adding "active" state to state machine
Bret Wortman
committed
37c5f47
First cut at new refreeze operation
Bret Wortman
committed
0481b62
Add default of refreeze all, with confirmation
Bret Wortman
committed
3da0254

Commits on Oct 20, 2025

Fix mounting repos in --check-status
Bret Wortman
committed
8eeb59d
Fixing index selection for mounting after thawing
Bret Wortman
committed
6ab4bce
--porcelain added throughout
Bret Wortman
committed
0aa8504
Output consistency
Bret Wortman
committed
6d6b136
Fix date range update logic in rotate
Bret Wortman
committed
70f531a
Start on new integration tests for thaw
Bret Wortman
committed
b79e27f

Commits on Oct 21, 2025

Fold & wrap text in Rich output
Bret Wortman
committed
3ef79b1
Add short option for porcelain
Bret Wortman
committed
cf6ef8f
Improved data stream detection during thaw operations
Bret Wortman
committed
07fbb93
Various pre-tests and other improvements
Bret Wortman
committed
935f19a
Check for plugin
Bret Wortman
committed
587eb26

Commits on Oct 23, 2025

Update progress display during --check-status
Bret Wortman
committed
7bb9ef5
Change default thaw duration from 7d to 30d
Bret Wortman
committed
20eb496
Updated thaw logic
Bret Wortman
committed
695238b
Updates to refreeze
Bret Wortman
committed
f99c8fa
Updating ILM processing for thawed indices
Bret Wortman
committed
1b2eb15
Default thaw list to only open thaws
Bret Wortman
committed
b88f7ba
Differentiate completed & refrozen
Bret Wortman
committed
cf60a93

Commits on Oct 27, 2025

Fixes to cleanup
Bret Wortman
committed
9519ad2

Commits on Oct 28, 2025

Linting & Pylance fixes
Bret Wortman
committed
401d91d
Make tests nondestructive
Bret Wortman
committed
52d410d

Commits on Oct 29, 2025

Fix object key vs object usage for s3 objects.
Bret Wortman
committed
ffd7c47
Change type hints
Bret Wortman
committed
0fcaed9
Integration tests
Bret Wortman
committed
a5c5ffe
Documentation
Bret Wortman
committed
9fb3895
Unit test fixes
Bret Wortman
committed
d015e32

Commits on Oct 30, 2025

Huge update after integration testing
Bret Wortman
committed
190fec7

Commits on Oct 31, 2025

Fixing integration tests in FAST_MODE
Bret Wortman
committed
1434ef9
Parallelize integration tests
Bret Wortman
committed
1d1aa27
Fixing repair-metadata
Bret Wortman
committed
66a0bc4
test fix
Bret Wortman
committed
813ef46
Speed improvements to thaw --check-status
Bret Wortman
committed
7d909bf
Implement --refrozen-retention-days on cleanup command
Bret Wortman
committed
c5fac67
Fixing cleanup of refrozen thaw requests
Bret Wortman
committed
990d3e7

Commits on Nov 3, 2025

Fixing refreeze logic
Bret Wortman
committed
8824711
Fixing error in refreeze
Bret Wortman
committed
338539b
Update legend
Bret Wortman
committed
0c5da57

Commits on Nov 4, 2025

Added short options
Bret Wortman
committed
1e4b6fc

Commits on Nov 10, 2025

Add fix for metadata repair
Bret Wortman
committed
443dbad
Final testing notes
Bret Wortman
committed
0be34ab