[ShopDB] Updates / Changelog

Discussion in 'General Discussion' started by Kozz, Dec 20, 2019.

  1. kkha

    kkha Norwegian Country Man
    ECC Benefactor Builder ⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Sep 26, 2020
    Messages:
    676
    Trophy Points:
    43,610
    Gender:
    Male
    Ratings:
    +150
    Deleted! (reason: quoted wrong)
     
    #161 kkha, Oct 5, 2020
    Last edited: Oct 5, 2020
  2. kkha

    kkha Norwegian Country Man
    ECC Benefactor Builder ⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Sep 26, 2020
    Messages:
    676
    Trophy Points:
    43,610
    Gender:
    Male
    Ratings:
    +150
    Agree!
     
  3. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    Development on hiding players inactive for 20+ days is nearly complete.
    I'm testing now to ensure it works as expected, and it should be released today.

    Once I'm finished testing I'll report back on what percentage of players are currently inactive, how many trade shop signs will be removed, etc.
     
  4. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    I've completed testing, and am about to release the new version.
    During testing, ~25% of items were from inactive players and were hidden.

    Out of 227 players:
    102 are inactive and will have their items hidden.
    135 are active and their items will remain shown.


    Out of 8,347 trade signs:
    1,953 signs are from inactive players and are now hidden.
    6,394 signs are from active players and will remain shown.

    I think this will greatly help to cleanse ShopDB of inactive shops.
    Here's an example that was sent to me by @black_kkha a couple of days ago:

    BEFORE -

    [​IMG]

    Every single player on that screenshot is inactive, and I believe all of their shops are out of stock as well.

    AFTER -

    [​IMG]

    This is much more realistic, showing only 3 players with active shops on main north selling coal.
    Also, @black_kkha is now accurately represented as having the best price. ;)

    I will be releasing these changes shortly!

    EDIT: It's also worth nothing that once an inactive player logs on again there items will not be hidden anymore.
    This process of determining who is / is not active is automated.
     
    #164 Kozz, Oct 10, 2020
    Last edited: Oct 10, 2020
  5. kkha

    kkha Norwegian Country Man
    ECC Benefactor Builder ⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Sep 26, 2020
    Messages:
    676
    Trophy Points:
    43,610
    Gender:
    Male
    Ratings:
    +150
    You're awesome @Kozz !

    Will it also hide shops who is empty over a period of example 72h? Or hide a shop that chest is full due to players selling to it, and not been emptied for a while?
     
  6. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    This change has been implemented now.
    This is just to hide inactive players ATM, but am looking into the possibility for a stock counter or ability to tell if a shop is out of stock ATM.
     
  7. kkha

    kkha Norwegian Country Man
    ECC Benefactor Builder ⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Sep 26, 2020
    Messages:
    676
    Trophy Points:
    43,610
    Gender:
    Male
    Ratings:
    +150
    Thanks mate! Please tell me when you have found out about the stock counting! :D
     
  8. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    It wasn't simple, but the time is nigh.
    I've made changes to the ChestShop plugin to add a stock counter, these changes were recently approved and added.

    I've been working to rebuild the back-end, front-end, and the crawler for ShopDB from the ground up to support these changes and make ShopDB even better. And yes, this includes a stock counter :)

    Here's a sneak-peek:

    [​IMG]

    * If you would rather have dark theme, I can change this. Just let me know.

    I've redesigned the crawler to be more reliable, and by the time this is released shops should be consistently updated every two to four hours.

    Many features will be initially lost, as I want to implement these changes ASAP .
    The following will not be included on release, but may be added later on:
    • Statistics panel will not be included on release - no recent searches, etc.
    • Shop Analytics will not be included on release - this needs re-work, anyways. Viewing chest shops by player/region will still be available
    • The bot will not be included on release, and the bot will be focused only towards teleporting players to chest shops.
    • 'Contact' section will be permenantly removed - all requests (new shops, bug reports, etc) will be made via these forums.
    • 'Helpers' panel will be (temporarily?) removed - adding regions is now automated, the only work that will have to be done is marking regions as active/inactive at a click of a button.

    ETA:
    Two weeks, hopefully. Within a month, definitely.
    ShopDB will remain online, but since I'm working on this, shops have not been and will not be updated until this is complete.

    Only signs with stock counters will be added to ShopDB. Stock counters are added automatically when the shop is used, or if the chest is opened/closed. So please update your shops by opening/closing the chests, or alternatively just wait for somebody to use them. :)
     
    • Winner Winner x 1
    • Friendly Friendly x 1
    • List
    #168 Kozz, Nov 1, 2020
    Last edited: Nov 1, 2020
  9. xHarambae

    xHarambae President
    President ⛰️⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ I ⭐ Premium Upgrade

    Joined:
    Mar 4, 2012
    Messages:
    1,465
    Trophy Points:
    70,810
    Gender:
    Male
    Ratings:
    +549
    Can you give a tutorial on how to properly create chestshops with the new update? I am unable to make one and I have it setup as goes; 1st line: xHaramabae, 2nd line: 3456, 3rd line: S 25920, and 4th line: Glowstone.
     
  10. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    (Resolved this in-game - not sure what the issue was)

    Chest shops are created the same way as you always have.
    You can, alternatively, include the stock counter when creating new chest shops - but this is unnecessary and will be immediately updated to the correct count anyways.
     
    #170 Kozz, Nov 1, 2020
    Last edited: Nov 1, 2020
  11. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    Quick update on progress.
    Went dark theme and tried to match the forum colors.

    Chest shops:

    [​IMG]

    Regions:

    [​IMG]

    Players:

    [​IMG]


    Schedule for automatic updates will be (server times):
    • Chest shops (signs and stock) - 10x per day
      • 12:00 AM
      • 04:00 AM
      • 08:00 AM
      • 10:00 AM
      • 12:00 PM
      • 02:00 PM
      • 04:00 PM
      • 06:00 PM
      • 08:00 PM
      • 10:00 PM
    • Regions (automatically adding any new regions with a warp) - once per day
      • 02:00 AM
    • Players (updating last seen times) - once per day
      • 06:00 AM
    Everything is now automated, from adding new regions to updating shops.
    Only manual action will be to mark regions as active/inactive (shop or not shop), as only 'active' regions will be crawled and updated.

    Remaining work before release:
    • Individual regions page, view chest shops and mayors in a specific region.
    • Individual players page, view chest shops and towns owned by a specific player.
     
    • Winner Winner x 2
    • Informative Informative x 1
    • List
    #171 Kozz, Nov 5, 2020
    Last edited: Nov 5, 2020
  12. icbomber38

    icbomber38 I can't complain but sometimes I still do
    ECC Sponsor Builder ⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ III ⭐ Premium Upgrade

    Joined:
    May 25, 2015
    Messages:
    218
    Trophy Points:
    53,910
    Gender:
    Male
    Ratings:
    +107
    so great @Kozz when is the release?
     
  13. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
  14. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    Currently working on adding main-east to the crawler, as ME just came back online recently.
     
  15. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    I've found a couple of bugs I'm trying to fix before next release:
    • Bot randomly disconnects while scanning, without any error or reason.
      • I think this is a bug with the Mineflayer API. It appears the bot is polling the server every 30 seconds and somehow deciding it's lost connection with the server and gives up all hope. I've written an override so this check never takes place, as this check is not accurate. Need to test some more, but I think this resolved this issue.
      • This issue was causing some regions to not be updated, and chest shops have been more of a "may update every 2 hours". The update rate has been reliant thus far, but this should make sure it's even more reliable.
    • The last region (sorted alphabetically) on each server is never updated.
      • My apologies, zena and zeeland. I have no idea why these aren't updating, and I'll have to look into it. I have not been able to recreate the issue, so perhaps the fix I've made above will fix this as well.
      • Edit: Found out why this is happening, and I've fixed it.
    • Unchecking 'Hide Disabled Regions' only shows disabled regions.
      • This is a flaw in the code, an easy fix but low priority. I'll look into it "soon".

    I have added changes to support main-east that are ready to be released, but trying to hash out these issues first.
    Unfortunately next release will require down-time of 4-5 hours, because I've changed some things that require a fresh DB reload.
    This release will likely be tomorrow.
     
    #175 Kozz, Nov 12, 2020
    Last edited: Nov 12, 2020
  16. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    Releasing these changes now.
    Had a few hiccups, but got them resolved.

    Currently rerunning the spider to repopulate the DB, and ShopDB should be back online within the next four hours.
     
  17. Kozz

    Kozz Builder
    Builder ⛰️ Ex-President ⚒️⚒️ Gameplay Architect

    Joined:
    Jan 11, 2017
    Messages:
    272
    Trophy Points:
    28,460
    EcoDollars:
    $0
    Ratings:
    +430
    ShopDB is undergoing maintenance and some large changes!

    Instead of chest shops being updated by a bot, they will now updated via a server plugin.

    What does this mean?

    ShopDB will now be updated near-real time. Every 10 minutes, shops are updated.
    A true 'chest is full' indicator is now possible and included. The ability to hide BOTH out of stock AND full chests is now available.
    This also opens up the door for lots of new features that were previously not possible.
    When a shop application is filled out and accepted, all of the signs for that shop will instantly appear on ShopDB.

    This is quite a big change, and required all items to be cleared out.
    Shop owners, your items will show up again on ShopDB AFTER:
    - You have opted-in and applied for shops in your town to be shown (if you have already applied, you do not need to re-apply.)
    - You open/close the inventory of the shop.
    - Somebody uses your shop.

    If your shop was previously included on ShopDB, then the quickest way to have your signs show up again is by opening/closing each of your chest shops.
    After you do this, it will take a maximum of 10 minutes for your shops to appear once again on ShopDB.
    Alternatively, you can do nothing and once players use your shop, it will also start appearing on ShopDB.

    As with any change, this could create new bugs. So please report them here if you find any!
    I'm keeping an eye out for any potential issues.

    I think this is a very good change, and am confident it will make ShopDB much more real-time and reliable.
     
    • Winner Winner x 13
    • Like Like x 1
    • List
    #177 Kozz, Jan 14, 2021
    Last edited: Jan 14, 2021
  18. Fr0zenTiger

    Fr0zenTiger Mayor
    Mayor ⛰️⛰️ Ex-Tycoon ⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Jan 2, 2019
    Messages:
    699
    Trophy Points:
    38,660
    Gender:
    Male
    EcoDollars:
    $4,068,811
    Ratings:
    +694
    Awesome :O
    Thanks for all the hard work you're putting into shopdb!
     
  19. techy88

    techy88 Tech-savvy Nub
    President ⛰️⛰️ Ex-EcoLegend ⚜️⚜️⚜️⚜️ Prestige ⭐ II ⭐ Premium Upgrade

    Joined:
    Sep 10, 2012
    Messages:
    542
    Trophy Points:
    66,360
    Gender:
    Male
    Ratings:
    +262
    Thank-you Kozz for all your work on shopdb. It is by far my favorite QoL addition to ECC in a long while. :)
     
    #179 techy88, Jan 14, 2021
    Last edited: Jan 14, 2021
  20. rh2wilson

    rh2wilson Tycoon
    Tycoon ⛰️⛰️⛰️ Ex-EcoLeader ⚜️⚜️⚜️ Ex-President ⚒️⚒️

    Joined:
    Jan 21, 2017
    Messages:
    227
    Trophy Points:
    31,810
    Gender:
    Male
    Ratings:
    +91
    ShopDB is the now officially the greatest thing about ECC.

    Thanks Kozz!