Hello!
Maps is a special case, being super gigantic[0] and basically on its own
private NFS server already, so I'm not sure we're ready to move its
storage onto Cinder. That said, I'm going to answer your questions here
for others who are following along:
- Is resizing straightforward?
Expanding storage pretty simple, there's a Horizon UI for resizing the
volume and then you'd need to run resize2fs to expand the file system.
Shrinking is not something we're likely to support.
- Will upfront allocation tie up resources?
Mostly not, cinder volumes are sparse so shouldn't consume physical
resources until the space is consumed with actual files.
- Is the performance of Cinder substantially better than the
/data/projects NFS?
We don't have a lot of experience to go on yet, but it should be much
faster: roughly the same as any other 'local' VM storage as it's backed
by the same Ceph cluster.
If you have a known non-terrifyingly-big subset of your data that you'd
like to move off of NFS, please open a phabricator quota request and
we'll see what we can do!
[0] So big that I don't know how big it is because I ran 'du' and gave
up after it sat there for 40 minutes
On 2/5/21 5:51 PM, Daniel Schwen wrote:
Neat,
I see the maps project has a 10GB quota on volumes. The tiles for the
WikiMiniAtlas take up considerably more space and are currently held
on /data/project. The appeal of /data/project is that it is a
"infinite" pit to store data in, whereas it seems that with Cinder
volumes I'd have to commit to a certain size a priori. Is resizing
straightforward? Will upfront allocation tie up resources? Is the
performance of Cinder substantially better than the /data/projects NFS?
Cheers,
Daniel
On Fri, Feb 5, 2021 at 4:32 PM Amir Sarabadani <ladsgroup(a)gmail.com
<mailto:ladsgroup@gmail.com>> wrote:
That's amazing!!!! Thank you so much.
On Fri, Feb 5, 2021 at 5:02 PM Andrew Bogott
<abogott(a)wikimedia.org <mailto:abogott@wikimedia.org>> wrote:
Wikimedia Cloud Services now supports attachable block storage
via the
OpenStack Cinder project. Attachable block storage is a
flexible storage
option that allows you to create volumes local to your project
but not
coupled to a particular VM; they can be moved between different
instances and persist after their associated volume is
deleted. Project
admins can access this feature via the 'Volumes' tab in Horizon.
I encourage all of you to start using Cinder storage for your new
databases and large data sets. Over the next few months we'll
be working
to move various use cases onto Cinder volumes and off of NFS
or LVM;
soon I hope to deprecate large-storage flavor types entirely
and support
all new non-root file storage with Cinder.
The default storage quota is quite small, but we plan to be
generous
with quota increases. To request additional storage, open a
phabricator
ticket here:
https://phabricator.wikimedia.org/project/view/2880/
<https://phabricator.wikimedia.org/project/view/2880/>
For more details about this feature, I've written a blog post,
here:
https://techblog.wikimedia.org/2021/02/05/cinder-on-cloud-vps/
<https://techblog.wikimedia.org/2021/02/05/cinder-on-cloud-vps/>
And, technical documentation can be found here:
https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_ins…
<https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_instances#Cinder:_Attachable_Block_Storage_for_cloud-vps>
Like any new feature, our implementation almost certainly
includes bugs
and missteps. Please provide feedback or feature requests via
phabricator or on the cloud mailing list.
-Andrew + the WMCS Team
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org
<mailto:Cloud-announce@lists.wikimedia.org> (formerly
labs-announce(a)lists.wikimedia.org
<mailto:labs-announce@lists.wikimedia.org>)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
<https://lists.wikimedia.org/mailman/listinfo/cloud-announce>
_______________________________________________
Wikimedia Cloud Services mailing list
Cloud(a)lists.wikimedia.org <mailto:Cloud@lists.wikimedia.org>
(formerly labs-l(a)lists.wikimedia.org
<mailto:labs-l@lists.wikimedia.org>)
https://lists.wikimedia.org/mailman/listinfo/cloud
<https://lists.wikimedia.org/mailman/listinfo/cloud>
--
Amir (he/him)
_______________________________________________
Wikimedia Cloud Services mailing list
Cloud(a)lists.wikimedia.org <mailto:Cloud@lists.wikimedia.org>
(formerly labs-l(a)lists.wikimedia.org
<mailto:labs-l@lists.wikimedia.org>)
https://lists.wikimedia.org/mailman/listinfo/cloud
<https://lists.wikimedia.org/mailman/listinfo/cloud>
_______________________________________________
Wikimedia Cloud Services mailing list
Cloud(a)lists.wikimedia.org (formerly labs-l(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud