[Software] Brickficiency - Find the cheapest BL stores to buy parts

[mdoupe]

Informal poll:

I'm trying to put together something for the report if no matches are found. For example, it finishes with the 5 store combos, and doesn't find any, so instead of just showing you a lot of nothing, it shows something at least partly helpful.

I was originally thinking to show a list of stores sorted by the raw number of lots or parts they could provide, however that's already available on Bricklink.

Then I was thinking that it might be good to help you eliminate some of the rare parts from the list. Perhaps a list of stores sorted by how many of the rarest parts they have from your list. I'm not sure how many items to count in this list though? The top 25% most rare items on the list? The top 10? What if there is only 8 parts on your list?

Let me know what you guys think.

I have the next version mostly ready to go, but part of it is that I have created a database much like BrickStore/BrickStock uses. What's holding it up is that distributing the database separately is against the Bricklink ToS. I've been trying to get them to let me do it under the same agreement they have with Patrick, but I'm getting a whole lot of nothing from the Bricklink CS team. I don't really want to revert back to the previous version's source, as there were quite a lot of changes made to change to using a database versus how it worked before. I also don't want to just go ahead and publish it because I have a somewhat large Bricklink store and I'd rather not be banned. :P

[legolijntje]

Well, showing what store has the most parts could be useful, so you don't have to go to Bricklink again. And, it's already filtered on the countries the user has selected.

About the database... Well, yea, to be honest, I would've expected such a reaction from Bricklink. Can you add a button in the program to download the database, instead of shipping it with it?

[Calabar]

The idea of legolijntje is good. The database changes continuously, include it in the software would no be useful. A button to download the database would allow an updated database and go round the problem of the bricklink ToS.

About "rare objects", maybe the best way would be to make them optional unless the uses select some kind of "I want it absolutely", or a "give it hight priority" flag. Maybe all the request could be managed using a (manual) "priority" system, where I assign a priority value to my parts.

Besides I think that the user should have the number of results he requested, regardless of the fact that all the parts has been retrieved or not. If I request 5 results, I will obtain the best five results even if no one of that will completely satisfy my wantedlist(s).

Another useful thing would be to manage alternate parts. For example I could need a brick and there are two suitable colours (a black chain, or a dark gray one), or I could need a brick that has some similar alternatives that are good for me too (a 6x6 dish 44375 or 44375b is the same for me), or I need a part in a rare colour but I can but I can settle for to buy one of a similar colour (a dark red angular 3x3 slope could be good in red too).

[mdoupe]

The database isn't distributed with the program, it's packaged up separately and downloaded the first time you launch the program. The issue is, it now includes thousands of set databases which take a long time to download. I download them and pack them all into a single compressed database, but this takes quite a long time, so it's unrealistic to have each user download each set inventory.

The issue is, distributing a copy of the database is against the Bricklink ToS.

[Lego Otaku]

I would like to see partial list if it can't find any combination at all. One list for parts that are rare (too few stores for example) so someone could separate the rare parts from the rest of the list and try to get the rest cheapest way while gathering rare part individually if needed.

[gezuvor]

I was going to suggest the "these parts are rare" results, but there is a better way to sort that out. When I first run the tool, I watch the store count closely as it scrolls up the screen. Once it's done, I stop the software and go back to remove the ones that show up less than 7-8 places from the search, preventing them from dragging the results down. If there were an intermediate step between the part-to-store count and the optimized search, it would be great. Something like "The following XX parts are available in less than YY stores. Do you want to omit them?" Alternatively, perhaps a config option that says "ignore parts found in fewer than X stores".

After the "common part" search is over, maybe a second search could be done that checks each of the stores in your results against the rare parts that were held back? Sort of a "BTW, you can get the unobtainium metallic silver wheel from mdoupe too." or something similar...

BTW - Small gripe. The last time I executed a search, the software ran for 3 days and tried 1.5 BILLION combinations. I'm thinking that if it takes more than a few hundred million combinations, it's trying too hard...

Edited July 9, 2014 by gezuvor

[mdoupe]

BTW - Small gripe. The last time I executed a search, the software ran for 3 days and tried 1.5 BILLION combinations. I'm thinking that if it takes more than a few hundred million combinations, it's trying too hard...

What can I say, it's thorough. :) I'm not really sure how to determine which routes are dead ends. Usually it only takes a long time if there are no rare parts in the search. (for example, if you're just searching for a bunch of plain bricks)

Please note that you can cancel the search at any time and it'll give you whatever matches it's found so far.

[Lego Otaku]

Avoid using 5 store searches. If you have a really large list, it may be easier to split them in 2. Do a search (with 1 store only setting) and it should show you a list of parts with number of stores that carried it. Take out the top few rarest pieces and hand search for it, it may be cheaper to order rare part individually. I've had case when I needed 10 of each and where rare part #1 was available in store A for $2 and store B for $5 each while rare part #2 was available in store B for $3 and store C for 50 cents. If you tried to order from store B only, you'd pay a lot more than from A and C (yes, after shipping).

No software can replace human capacity for getting the best deal but the software can do the bulk of the work quickly if you don't have too many hard to find pieces.

[Heppeng]

Its a good point, I often find that when a two store solution is found, it is worth checking a three store solution because the three store solution is often cheaper than the difference in the extra postage required. I suspect that it would be somewhat complex for brickficiency to include postage costs in the search, if it is indeed possible at all! It would also be helpful if solutions where the cost of a few bricks from one store is below their minimum order value there could be an option to disregard that solution, but it is not too hard to go through the list manually to do that.

[Lego Otaku]

The store limit and fees are usually spelled out in the term pages and not stored in a manner that would make it easy for Brickficiency to look it up and use it. With hundreds store around and so many variations of limits and fees, it is probably not going to be easy to set up BF to process them. Not to mention time consuming for someone to check hundreds store and entering the fees and limits.

Shipping is another mess to process as well. Fee varies by countries, by how well seller packs it (very well packed can weight more and go into next fee bracket), and by carrier used. Taxes and duties is also no fun to figure out but usually one would have an idea how much to expect if ordering from outside the home country.

[Zimix]

Nice program. I tried to use it. And I have some ideas and questions.

1. I should be able to limit the shops with help of my least favorit store list.

2. The search, should first check if an item is only available in some shops, even for a qty less. And then try to find a solution. I don't know how it does it now.

3. Is an item only taken from one shop in the solutions ? or can it be split over several shops (due to one shop not having enough stock) to get to the needed qty.

4. Why can we not go over 5 maximum combination to query ?

5. In some country's, it's sometimes better to have small orders, but more shipment due to import duties sometimes not applied to small amounts. So solution with min amount of shops is not always the cheapest.

Hope this can help

And again great program

[mdoupe]

1. I should be able to limit the shops with help of my least favorit store list.

2. The search, should first check if an item is only available in some shops, even for a qty less. And then try to find a solution. I don't know how it does it now.

3. Is an item only taken from one shop in the solutions ? or can it be split over several shops (due to one shop not having enough stock) to get to the needed qty.

4. Why can we not go over 5 maximum combination to query ?

5. In some country's, it's sometimes better to have small orders, but more shipment due to import duties sometimes not applied to small amounts. So solution with min amount of shops is not always the cheapest.

1. good idea, I'll add it to the list

2. the search considers all shops an item is available from, even if the shop only has one of the item.

3. it splits it over several shops, if that is the cheapest way to do it.

4. the amount of time searches take goes up exponentially. I'll probably bump it up to 6 in the future, but some lists already take DAYS to calculate at 5.

5. This is what the "Continue looking for larger combinations" checkbox is for. Also, note: if you don't like, say, the 2 store combinations that the search found you can immediately do another calculation with the cached price guide results. It does not have to download the page again until the next time you launch the program.

[Zimix]

Thank you for help and info

[Lego Otaku]

BF already stops calculation before it starts if there is no source of a part you have on want list.

Occasionally the part gets imported wrong. Usually it loses color information and if I manually set color, it works. If I have really rare part on my list or a part that doesn't exist in color I want, it'll stop and not waste hours looking for stores.

[Zimix]

There should be a way to enable enforcing "minimum buy" per shop or not. as a tick box.

[mdoupe]

There should be a way to enable enforcing "minimum buy" per shop or not. as a tick box.

I'd love that option too. Unfortunately, the only way (that I'm aware) to find that info is to go to the store policy page. Retrieving thousands of store policy pages would take quite a long time (and might anger Bricklink in the process).

[Zimix]

How if the user fills a table with the minimum buy for his prefered shops, and he could add the minimum buy of those shop proposed by the program. It's not perfect but could work.

[Calabar]

I'd love that option too. Unfortunately, the only way (that I'm aware) to find that info is to go to the store policy page. Retrieving thousands of store policy pages would take quite a long time (and might anger Bricklink in the process).

Are you sure? I think that the minimum buy is a datum stored in the table that contains the summary for the shops.

In fact it is one of the data clearly showed when you search for an item, or you choose "Show All" page for your wanted list.

Edited July 15, 2014 by Calabar

[Lego Otaku]

It can be tedious to list hundred stores with minimum buys into BF, especially when you don't know which stores are going to be picked among the cheapest. Plus they can change the term or limit without notice so if you set up BF then in between time you last checked and entered the number and the time it spits out the list, and the time you get to buying the parts the limit can change.

You're better off manually checking the few stores for limits and fees before you commit to filling the shopping cart and buying them.

[Lego Otaku]

I think my copy is being goofy. I've restarted Brickficiency a few times and tried again, it kept reporting no match found (even 5 combinations) and when I had it display the part list, this came up:

-
Parts list sorted by number of stores
0 stores have White Plate 1 x 2

I checked Bricklink, there are more than 900 sources and stock of over 1 million 1x2 white plate part # 3023 yet every time I tried, Brickficiency seems to think there is no source.

[Pat-Ard]

Hi folks,

since a couple of days I only get errors when conecting to BL for reading the wanted lists. When will this be fixed?

Pat

[mdoupe]

since a couple of days I only get errors when conecting to BL for reading the wanted lists. When will this be fixed?

Not sure. it's working for me with both the latest release and my current development version. Can you send me Appdata\Brickficiency\debug\Debug-import.txt ?

[Pat-Ard]

sent you an PM.

Pat

[JGW3000]

First, thank you for providing this awesome app, it really helps

Is there any way to get some decorated parts into the searchable parts list, this part was not added to the search list although it was in the imported want list: http://www.bricklink.com/catalogItem.asp?P=3070bpb077

[mdoupe]

First, thank you for providing this awesome app, it really helps

Is there any way to get some decorated parts into the searchable parts list, this part was not added to the search list although it was in the imported want list: http://www.bricklink...sp?P=3070bpb077

Decorated parts are not differentiated in any way, it only searches using the item ID. Make sure your Bricklink database is up to date (Tools > Update Database). Make sure the part is showing up Reddish brown in the list (that's the only colour it's available in). Also make sure you're searching in a country that has one for sale. It's quite rare and only available in a few countries.