This project has moved. For the latest updates, please go here.

What I'd like to see

Sep 5, 2010 at 6:08 PM

1) I'd like to see a plan oriented to the end user not the developer. People need to know what is coming and roughly when (or after what).

2) I'd like more people in here. :D (I plan to start advertising this today with my facebook account etc.)

3) I'd like settings to be as much stored redundantly as possible. In fact I would try to store: In "all users", in registry (although I hate the concept) AND in all involved disks root. This would help recovery. In fact in a nice system, the system would find the newest copy and fix things based on that. Like in pro RAID systems where the RAID controller stores config in its flash but also on hidden blocks on all the disks.

4) Parity and "JBOD" (i.e. Liquesce) should remain two distinct projects.

5) I'd like to see the live parity part of the equation too. :P Yes my trust on that other project is hurt seriously (for now).

6) For Liquesce my wishlist:

(a) #3 above

(b) Smart AND user selectable file/folder distribution. I expect to see a choice of filling disks one by one (possibly up to a threshold - and with use choice "smaller-to-larger" or the opposite or even following a specific priority set by the user) OR evenly filling them AND allowing at all times the user to put something on a specific disk manually without messing Liquesce's logic.

(c) The logic above to expand with "green" philosophy. A single folder should remain on one disk if possible. Other disks don't need to spin if files the user is looking (like "Music") are all together. In fact NO disk should spin if the system includes a nice smart directory listing cache (user configurable size and supporting maintenance in case something de-syncs it... manually putting a file on a specific disk shouldn't normally de-sync it, as the OS has functions that can alert Liquesce of the change).

(d) On a more mature system, a maintenance mode where the system will be able to move files around based on the logic of "b" above. In other words if the user decided in the past to evenly fill disks and then changed his mind to fill them one by one, to be able to do it. "Safe" moves of course. The same maintenance mode could also become smarter to support "c" green philosophy. For example system had decided that folder "videos" and "pictures" are on the same disk (user has decided to fill disks one by one). Then adding to videos makes this folder so large that "pictures" wouldn't fit if videos where all in the same disk. So the system puts new videos in the next disk but when in maintenance, shifts the whole "pictures" folder on next disk and puts all videos together on the first. It's a wishlist, ain't it? :) Of course this isn't very "green" as it creates more disk I/O daily (or whenever system is doing maintenance). But give user the choice of what happens during maintenance.

(e) Liquesce views should have their own user/group security and of course be able to be mapped to a drive letter and shared (else what is the point eh?).

(f) Should be able to work on various FS (in fact if possible not care for the FS as long as normal system calls work) - in fact even different FS at the same time.

(g) Folder caching should also support in the future, some off-line mode. In other words set some attribute for files on a missing disk and still show them (in fact it should be user choice). This will possibly help cooperation with parity systems.

(h) When the system matures, to automatically support something like Windows 7 "library" logic. i.e. drop a file on a specific folder in a Liquesce view based on file type.

That's all coming to me for now. I can build a more organized wishlist (along with other people's ones) if needed.

 

Coordinator
Sep 5, 2010 at 10:00 PM

Wow

I'll try to answer some of these tomorrow night.

Might have been better split into individual things, but that I suppose will depend onthe answers, and When your on a roll, you sure do go for it?

Much appreciated. and will help keep me focused onthe user side, rather than just on the challenge side of things.

Anyone else want to add some "Requirements"?

Sep 6, 2010 at 8:58 AM
smurfiv wrote:When your on a roll, you sure do go for it?
That more or less describes me exactly. :)
Sep 6, 2010 at 8:59 AM

I plan to clear this a bit, make it more generic and put it on my blog.

...that, because I plan on posting the link on that other forum too (not that I expect my wishlist to be considered, but anyway).

 

Coordinator
Sep 6, 2010 at 6:42 PM

Answers in Blue

1) I'd like to see a plan oriented to the end user not the developer. People need to know what is coming and roughly when (or after what).

  • OK. I was going to at least get the Application tray done, (From the asiprations home page) and then see what features have been requested.
  • These would then form the "Plan" for the next "big" phase

2) I'd like more people in here. :D (I plan to start advertising this today with my facebook account etc.)

  • Cool
  • So would I.. GUI's with WPF are not my strong points (Yet)

3) I'd like settings to be as much stored redundantly as possible. In fact I would try to store: In "all users", in registry (although I hate the concept) AND in all involved disks root. This would help recovery. In fact in a nice system, the system would find the newest copy and fix things based on that. Like in pro RAID systems where the RAID controller stores config in its flash but also on hidden blocks on all the disks.

  • Already storing the config file in the All Users area under "$CommandApplicationData\LiquesceSvc", this is hidden by default. Under Win7 it hides in here C:\ProgramData\LiquesceSvc.
  • The config data is in XML format so is easy to see what is going on, and wil be easy to midify
  • The Logs from the Service go under sub-directory, Called logs
  • The rest I think is more parity stuff

4) Parity and "JBOD" (i.e. Liquesce) should remain two distinct projects.

  • Agreed
  • I do not see (currently) why the driver and services that "JBOD" should be used for the Parity.
  • They need to operate independently.

5) I'd like to see the live parity part of the equation too. :P Yes my trust on that other project is hurt seriously (for now).

  • I haven't forgotten...

6) For Liquesce my wishlist:

(a) #3 above

  • Apart from the Raid / Parity stuff !

(b) Smart AND user selectable file/folder distribution. I expect to see a choice of filling disks one by one (possibly up to a threshold - and with use choice "smaller-to-larger" or the opposite or even following a specific priority set by the user) OR evenly filling them AND allowing at all times the user to put something on a specific disk manually without messing Liquesce's logic.

  • The user can place what they want where they want.
  • The rest would be messing Liquesce's logic

(c) The logic above to expand with "green" philosophy. A single folder should remain on one disk if possible. Other disks don't need to spin if files the user is looking (like "Music") are all together. In fact NO disk should spin if the system includes a nice smart directory listing cache (user configurable size and supporting maintenance in case something de-syncs it... manually putting a file on a specific disk shouldn't normally de-sync it, as the OS has functions that can alert Liquesce of the change).

  • That's how it attempts to work at the moment.
  • The caveat comes when it needs to create new directory
    • It will iterate over the drives in the order the user has selected them
    • It will then see if there is enough space (See HoldOffBuffer size in the Advanced settings) for the creation
    • It will then "Replicate" the directory structure to allow the creation of the new file / directory
  • It will only access the drives that actually contain the dat it has "previously" been told to find. (So no Filewatcher classes !)
  • It will not scan the drives before opening and store the data (Thus a drive will stay asleep if nothing else "talks" to it)

(d) On a more mature system, a maintenance mode where the system will be able to move files around based on the logic of "b" above. In other words if the user decided in the past to evenly fill disks and then changed his mind to fill them one by one, to be able to do it. "Safe" moves of course. The same maintenance mode could also become smarter to support "c" green philosophy. For example system had decided that folder "videos" and "pictures" are on the same disk (user has decided to fill disks one by one). Then adding to videos makes this folder so large that "pictures" wouldn't fit if videos where all in the same disk. So the system puts new videos in the next disk but when in maintenance, shifts the whole "pictures" folder on next disk and puts all videos together on the first. It's a wishlist, ain't it? :) Of course this isn't very "green" as it creates more disk I/O daily (or whenever system is doing maintenance). But give user the choice of what happens during maintenance.

  • Sounds like something for V 2 development phase

(e) Liquesce views should have their own user/group security and of course be able to be mapped to a drive letter and shared (else what is the point eh?).

  • Currently - Whatever the system has set up for access, should be replicated when the service restarts
  • This is why I chose to use the LanManServer to perform the security share refresh.
  • It means that whatever the system has implemented will, just the same, and I do not have to do anything clever with ACL's and security access permissions on registry hives etc.

(f) Should be able to work on various FS (in fact if possible not care for the FS as long as normal system calls work) - in fact even different FS at the same time.

  • If the Windows OS can read / write to it.
  • Then according to the MSDN, this should be able to as well.
  • I have NTFS (3 / 4) and exFat currently being used.

(g) Folder caching should also support in the future, some off-line mode. In other words set some attribute for files on a missing disk and still show them (in fact it should be user choice). This will possibly help cooperation with parity systems.

  • I'm sure that is just asking for trouble, If the OS cannot see it, then it should not be made visible for anything to attempt to see it.
  • The OS is always trying to do clever things (Like get Frame at position 3 Mins to show the Icon etc )

(h) When the system matures, to automatically support something like Windows 7 "library" logic. i.e. drop a file on a specific folder in a Liquesce view based on file type.

  • I think I missed something there..

That's all coming to me for now. I can build a more organized wishlist (along with other people's ones) if needed.

  • If you need to have a better "Drill-Down / Explanation" of anything above, then please split them out to seperate discussions.

 

Thanks for the ideas, and keeping on track :-)

Sep 6, 2010 at 7:27 PM
And mine in Red :D
smurfiv wrote:

Answers in Blue

1) I'd like to see a plan oriented to the end user not the developer. People need to know what is coming and roughly when (or after what).

  • OK. I was going to at least get the Application tray done, (From the asiprations home page) and then see what features have been requested.
  • These would then form the "Plan" for the next "big" phase
  • GREAT

2) I'd like more people in here. :D (I plan to start advertising this today with my facebook account etc.)

  • Cool
  • So would I.. GUI's with WPF are not my strong points (Yet)

3) I'd like settings to be as much stored redundantly as possible. In fact I would try to store: In "all users", in registry (although I hate the concept) AND in all involved disks root. This would help recovery. In fact in a nice system, the system would find the newest copy and fix things based on that. Like in pro RAID systems where the RAID controller stores config in its flash but also on hidden blocks on all the disks.

  • Already storing the config file in the All Users area under "$CommandApplicationData\LiquesceSvc", this is hidden by default. Under Win7 it hides in here C:\ProgramData\LiquesceSvc.
  • The config data is in XML format so is easy to see what is going on, and wil be easy to midify
  • The Logs from the Service go under sub-directory, Called logs
  • The rest I think is more parity stuff
  • Side note: Aurora and Veil (just installed both on VM) already do exactly what I asked. You find the disk configuration XML in all disks (!). Nice.

4) Parity and "JBOD" (i.e. Liquesce) should remain two distinct projects.

  • Agreed
  • I do not see (currently) why the driver and services that "JBOD" should be used for the Parity.
  • They need to operate independently.

5) I'd like to see the live parity part of the equation too. :P Yes my trust on that other project is hurt seriously (for now).

  • I haven't forgotten...

6) For Liquesce my wishlist:

(a) #3 above

  • Apart from the Raid / Parity stuff !

(b) Smart AND user selectable file/folder distribution. I expect to see a choice of filling disks one by one (possibly up to a threshold - and with use choice "smaller-to-larger" or the opposite or even following a specific priority set by the user) OR evenly filling them AND allowing at all times the user to put something on a specific disk manually without messing Liquesce's logic.

  • The user can place what they want where they want.
  • The rest would be messing Liquesce's logic
  • Elaborate please?

(c) The logic above to expand with "green" philosophy. A single folder should remain on one disk if possible. Other disks don't need to spin if files the user is looking (like "Music") are all together. In fact NO disk should spin if the system includes a nice smart directory listing cache (user configurable size and supporting maintenance in case something de-syncs it... manually putting a file on a specific disk shouldn't normally de-sync it, as the OS has functions that can alert Liquesce of the change).

  • That's how it attempts to work at the moment.
  • The caveat comes when it needs to create new directory
    • It will iterate over the drives in the order the user has selected them
    • It will then see if there is enough space (See HoldOffBuffer size in the Advanced settings) for the creation
    • It will then "Replicate" the directory structure to allow the creation of the new file / directory
  • It will only access the drives that actually contain the dat it has "previously" been told to find. (So no Filewatcher classes !)
  • It will not scan the drives before opening and store the data (Thus a drive will stay asleep if nothing else "talks" to it)
  • OK interesting

(d) On a more mature system, a maintenance mode where the system will be able to move files around based on the logic of "b" above. In other words if the user decided in the past to evenly fill disks and then changed his mind to fill them one by one, to be able to do it. "Safe" moves of course. The same maintenance mode could also become smarter to support "c" green philosophy. For example system had decided that folder "videos" and "pictures" are on the same disk (user has decided to fill disks one by one). Then adding to videos makes this folder so large that "pictures" wouldn't fit if videos where all in the same disk. So the system puts new videos in the next disk but when in maintenance, shifts the whole "pictures" folder on next disk and puts all videos together on the first. It's a wishlist, ain't it? :) Of course this isn't very "green" as it creates more disk I/O daily (or whenever system is doing maintenance). But give user the choice of what happens during maintenance.

  • Sounds like something for V 2 development phase

(e) Liquesce views should have their own user/group security and of course be able to be mapped to a drive letter and shared (else what is the point eh?).

  • Currently - Whatever the system has set up for access, should be replicated when the service restarts
  • This is why I chose to use the LanManServer to perform the security share refresh.
  • It means that whatever the system has implemented will, just the same, and I do not have to do anything clever with ACL's and security access permissions on registry hives etc.

(f) Should be able to work on various FS (in fact if possible not care for the FS as long as normal system calls work) - in fact even different FS at the same time.

  • If the Windows OS can read / write to it.
  • Then according to the MSDN, this should be able to as well.
  • I have NTFS (3 / 4) and exFat currently being used.
  • Great.

(g) Folder caching should also support in the future, some off-line mode. In other words set some attribute for files on a missing disk and still show them (in fact it should be user choice). This will possibly help cooperation with parity systems.

  • I'm sure that is just asking for trouble, If the OS cannot see it, then it should not be made visible for anything to attempt to see it. Except if it prompts you to bring online the appropriate device (like asking for a missing CD).
  • The OS is always trying to do clever things (Like get Frame at position 3 Mins to show the Icon etc )

(h) When the system matures, to automatically support something like Windows 7 "library" logic. i.e. drop a file on a specific folder in a Liquesce view based on file type.

  • I think I missed something there..
  • Yes I wasn't clear. I'll come back to that later.

That's all coming to me for now. I can build a more organized wishlist (along with other people's ones) if needed.

  • If you need to have a better "Drill-Down / Explanation" of anything above, then please split them out to seperate discussions.
  • I think it's early for that, ain't it?

 

Thanks for the ideas, and keeping on track :-)


And thanks for trying to make something useful for all of us.

 

Coordinator
Sep 6, 2010 at 8:45 PM
NLS wrote:
And mine in Red :D
smurfiv wrote:
That's all coming to me for now. I can build a more organized wishlist (along with other people's ones) if needed.
  • If you need to have a better "Drill-Down / Explanation" of anything above, then please split them out to seperate discussions.
  • I think it's early for that, ain't it?

Some of the above have been moved a new  document section [url:http://liquesce.codeplex.com/wikipage?title=FAQs]

Sep 8, 2010 at 8:12 PM
Edited Sep 8, 2010 at 8:18 PM

If you saw my recent post in the other forum, when/if someone makes a project about parity (and I mention this here because I already talk about parity), it might be

a) interesting

b) not very complex

c) very useful

d) different than the "other" solution

...to actually implement block based parity (i.e. ignoring the file system and reading the disks' binary data). Something like unRAID for Windows.

Since new Microsoft solutions like Veil and Aurora (esp. the first which will be commonplace for the enthusiasts) or use of Liquesce with any supported OS will actually solve the JBOD part, the parity part could be solved as above.

Esp. Veil and Aurora don't let you access distinct disk filesystems at all, there it could be the ONLY solution.

Hope someone gets interested.

Only challenges I can see are two:

- How the parity system will get notified that a write took place and where (to recalculate the parity for those blocks).

- What will happen with rapidly changing data (like video recording). Maybe parity should be recalculated only during file close and not true real time. Risky but could be ok for home use.